Single-token steering with the context-vector difference shifts the answer state toward the target, and behavior follows at the mid-stack layers where alignment peaks (MODERATE confidence)

Methodology: docs/methodology/issue_1415.md · gist

Takeaways

• Single-token patching of the context-vector difference moves the answer state toward the target: cosine 0.36–0.41 after correcting a shared-baseline inflation of ~0.08 (disjoint baseline halves), 28/28 pairs above their random-direction nulls on both arms; correcting further for split-half reliability puts the matched-layer alignment at 0.29–0.43; the traversal is 2–5% of the target norm.
• Position-resolved, the edit's two channels separate: the realized-text divergence is early-concentrated (early-vs-late log-ratio +0.71 to +1.47), but re-forwarding identical text with the hook re-armed shows the direct component persisting to the end of the span in all 4 primary cells — late bins clear twice the same-text jitter floor by 2.5–10× (111/112 pairs positive) while decaying 11–30× to 0.4–2% of the state norm — and baseline-text re-forwards (context arm, no interval stored) carry 93% of it at layer 20 and 65% at layer 14: a largely text-independent channel.
• Behavior follows the geometric layer profile: the judged shift peaks at the layer-14 context arm (+6.2 points, p = 0.008 — 21% of the context-swap ceiling, 7× the layer-20 primary read of +0.9) and replicates on two fresh seed bases (+6.6, p = 0.007; +5.0, p = 0.025), the terse-pair flip reproducing on both; layers 7–10 and the prefix arm stay near floor.
• The fitted context-to-answer map predicts none of the realized shift at layer 20 (mean cosine 0.00; magnitude over-predicted ~16×); other layers were not computed.
• Matched-query steering vectors transport far better than cross-query ones (0.49 vs 0.22, prefix arm, p = 8e-6), and the layer-14 behavioral movement is matched-query-only (+11.8 vs −0.3 cross); transportability is query-dependent, though closer matched targets could carry part of the gap.
• All-position steering flips 96–98% of draws into Chinese script (a broken output distribution, so its taller geometric bars are reads on corrupted text), and the medical-doctor pair's target is sampling noise (split-half 0.049); that pair is uninterpretable.

Goal

• This experiment in context: The context-to-answer mapping line (#685, #779, #823, #841, #922, #1092) is entirely correlational: probes, regressions, and teacher-forced predictability reads over the context vector (last-context-token residual activation) and the answer profile (mean answer-span activation). This run is the line's first causal test: patch a context's state with the difference between two contexts' vectors, generate on-policy, and measure whether the realized answer state moves toward the target context's answer profile, whether the fitted map from #922 predicts the movement, and whether judged behavior moves with it — against random-direction and shuffled-pair nulls, a context-swap ceiling, and the persona-vector steering baseline. #1092 predicts matched-query differences transport best; #823's content-identity caveat predicts the map may not transport.
• Broader narrative: This serves the context-as-a-vector question (docs/open_questions.md, spec-context-as-vector anchor): whether a context's effect on the model can be treated as a vector at a single position. It also gives the steering anchor (spec-steering) its first in-house causal datapoint: answer-state geometry and behavior dissociate under intervention, so correlational geometry reads do not license behavioral-steering claims.

Methodology

Design: Qwen/Qwen2.5-7B-Instruct (28 layers, hidden size 3584), no training. A bank of 28 context pairs (c, c′): 15 matched-query pairs — 10 instruction swaps (same user question; no system prompt vs one behavioral instruction, reused from the instruction-shift capture line) and 5 persona-condition swaps (same question set under different persona shot-count conditions, reused from the persona-monitoring condition bank) — plus 13 cross-query pairs (different questions under different personas from the same condition bank). The pair prompts and questions are LLM-generated synthetic corpora — data-realism tier 3 per the plan, reused verbatim because no established benchmark provides matched-query persona pairs aligned with this artifact line and reuse keeps the causal read comparable to the correlational parents. For each pair, the steering vector is the context-vector difference Δ = V_c(c′) − V_c(c), added by a forward hook to the residual stream at the last context token throughout generation. Both extraction arms run per the standing prefix/context rule: prefix-based (last token of the system/persona prefix) and context-based (last token of prefix + user query). Sweeps: steer layer in {7, 10, 14, 17, 20, 21, 24} with layer 20 primary; steering scale α in {0.5, 1, 2, 4} under a coherence gate; position last-context-token (primary) vs all generated positions. Arms: unhooked baseline under c; steered arms; context-swap ceiling (generate under c′ directly); norm-matched random-direction null (500 draws per pair, selection-symmetric); shuffled-pair null (other pairs' real steering vectors, norm-matched); persona-vector steering baseline (per-layer difference-of-means directions for evil, sycophancy, and hallucination, coherence-gated over the same α grid). Every compared arm runs on the same hooked-HF generate() stack — a stated deviation from the project vLLM rule, taken because hooks force HF for the treatment arm and a mixed-stack contrast would confound both DVs. N = 10 on-policy draws per cell at temperature 1.0.

Rounds:

Training: N/A — no model training. Generation, steering, and judge constants:

Evaluation: The geometric primary DV is the cosine between the realized answer shift (mean steered answer profile minus the baseline answer profile) and the target direction (answer profile of c′ minus answer profile of c), per read layer; the headline statistic is the mean over pairs of the max over the 7 steer layers at α = 4, each pair compared with its own selection-symmetric random-direction null band (97.5th percentile). The initially computed statistic shared the same 10-draw baseline mean between shift and target, adding a positive noise term the null cannot model; the corrected primary read splits the baseline draws into disjoint 5-draw halves (target from one half, shift from the other; both half-assignments computed and averaged). The two half-assignments agree to ±0.003 on the max-over-layers aggregate; the layer-20-anchored aggregates differ by up to 0.022 between splits. Because 5-draw halves double baseline-mean noise, the disjoint value is attenuated — the truth lies between the disjoint and shared conventions, and both are reported. The behavioral dual-DV is a graded 0–100 judge score (primary; per-pair rubric instantiated from the pair's trait or instruction, one behavior per call, drop-never-coerce) plus the binary rate companion; companion validation: Spearman ρ(graded, binary) = +0.41 across all 420 (arm, pair, α) cells, p = 1.2e-18, and +0.67 across the 63 cells with dynamic range. Behavioral judging covers the primary-layer and selected-α cells (4,200 items × 5 draws = 21,000 judge calls, Batch API, 0 transport losses); a follow-up round judged the layer-sweep steered cells at α = 4 under the identical recipe (3,360 items × 5 draws, separate work-dir and caches; per-arm content drops 0–10 and transport losses 3–9 of ~1,400 draws — under 1%, roughly arm-symmetric, carried as a caveat rather than re-judged); the persona-vector α-search cells (~600 completions) feed only the coherence gate (recorded deviation). The context-swap ceiling's geometric alignment is 1.0 by construction (a normalization anchor), so the partial-vs-ceiling read is anchored on the behavioral ceiling, which is empirical. The map-transport DV was computed at layer 20 only (recorded deviation; the reused artifact also carries maps at layers 15, 18, 21, 25, 27, of which one — 21 — is in this run's sweep). Instrument checks: target split-half reliability is 0.853–0.988 for 27 of 28 pairs; the medical-doctor pair sits at 0.049 (cleared the 0.043 null bar by 0.006) and is flagged as a noise target wherever it enters aggregates. Five of 28 pairs shift the ceiling judge score by less than 5 points (dead behavioral ceiling), diluting ceiling-anchored behavioral contrasts on those pairs. Judge content drops: baseline 0; steered 8–12 of 5,600 draws; ceiling 76 of 1,400 (concentrated on evil-persona target completions, most plausibly biasing the ceiling down — conservative for the steered-far-below-ceiling contrast). P-values in Results come from a one-sided unequal-variance two-sample test (matched-vs-cross, with a rank-sum companion) and a paired signed-rank test (steered-vs-baseline shifts). The replication round re-sampled the layer-14 α = 4 steered cells (both arms, 28 pairs) plus fresh unhooked baselines on seed bases 43 and 44 with the parent steering vectors frozen — a fresh-sampling-only design — judged under the identical graded recipe (840 items × 5 draws per seed; content drops ≤12 draws per arm, two rep-43 baseline items fully dropped, 0 transport losses).

The answer-position round re-forwards the stored completions teacher-forced with no steering hook — every measured forward runs the clean model on the realized text under the original context, so the per-bin quantities are position-resolved footprints of the realized TEXT divergence, not the generation-time state shift (the edited residual entry is absent from every measured forward; mechanism attribution between attention back to the edited position and changed-early-tokens self-conditioning is out of scope by design). Residual activations are pooled per answer-position bin (13 bins: first token, tokens 2–5, relative deciles 1–10, last token) at read layers 14 and 20 matched to the steer layer, α = 4 cells. Per (pair, arm, layer, bin): shift = steered minus baseline bin mean (disjoint 5-draw baseline halves, both assignments averaged — the parent's convention; replication rounds use all 10 rep draws against the parent target), target = context-swap ceiling minus baseline, noise floor = norm of the baseline half-mean difference. The headline statistic is the within-pair log-ratio of mean early-bin (first + tokens 2–5) to mean late-bin (deciles 8–10) shift norm, averaged over pairs with a 10,000-resample pair bootstrap and a signed-rank companion; companions: the same ratio with width-matched early bins (deciles 1–2), the same ratio over the noise floors (the noise-only reference — 7 pairs with a zero early floor drop out: their baseline halves share identical first tokens), exclude-medical and matched-length sensitivity reads (both move the headline under 0.03), and a per-bin random-direction null (500 draws). Span-level alignment is additionally disattenuated: the disjoint cosine divided by the root product of split-half reliabilities of shift and target (Spearman–Brown-adjusted, k = 4/3; corrected values reported only where both reliabilities clear 0.1; the medical-doctor noise target excluded; draw and pair bootstraps). A parity gate recomputed plain span means for 10 spot cells against the parent's stored tensors before the sweep: 10 of 10 within cosine 0.9999. Length confound: the terse pair's steered context-arm answers run 12–44 tokens against a 554-token baseline median, compressing its relative-decile contrast — its early-vs-late ratio ranks 22nd of 28 in the layer-14 context cell against a predicted panel maximum; the formal pair's lengths are unchanged (452–607 steered).

The hooked-unhooked decomposition round re-forwards the same stored completions teacher-forced with the parent steering hook re-armed (α = 4 at the last context token, steer layer 14 or 20), so the per-bin hooked-minus-unhooked difference on identical text isolates the generation-time direct component — the effect that remains with the realized text held fixed — which the unhooked answer-position round could not separate from text-mediated self-conditioning. 168 hooked capture cells: 28 pairs × {steered text with the context difference vector, steered text with the prefix difference vector, baseline text with the context difference vector} × steer layer, captured at layers {7, 10, 14, 17, 20, 21, 24} in fp16 (bf16 forwards); the unhooked side reuses the answer-position round's 336 shards. Because the hook edits only the context position, the steer layer itself is causally zero on the answer span and serves as the same-text repeat-precision (jitter) reference; the re-forwards were bit-deterministic there (hooked-minus-unhooked magnitude 0 at the causally-zero layer in all 6,552 (pair, bin) rows), so the band is the two-sided fp16 storage-quantization floor in all 2,184 headline (pair, bin) cells — a floor on measurement precision rather than an empirical noise estimate, conservative for the persistence read: the weakest primary cell clears at interval-low +0.71 = ln(2.04), so the verdict survives a band under-estimate up to ~2×, and the bit-determinism rules a genuinely larger true band out. Per (pair, bin), the direct magnitude is the norm of the mean hooked-minus-unhooked difference over the 10 draws; the per-pair headline statistic is the log of mean late-bin (deciles 8–10) magnitude over twice the jitter, and the cell verdict is its pair mean with a 10,000-resample pair bootstrap (seed 14153) and a signed-rank companion — direct-persistence iff the mean is positive with the 95% interval excluding zero, late-null iff the interval sits wholly below zero (the plan v11 §7 lattice, fixed before the run; its falsification clause required late-null in all four primary cells). Read layers were fixed in plan v11 §5 and never maxed over: steer 14 → headline read 17 (20/21/24 side by side), steer 20 → headline read 21 (24 side by side). Companions: early-vs-late log-ratio with early = {first token, tokens 2–5} and the width-matched {deciles 1–2} variant; per-bin cosine of the difference to the counterfactual target against a 500-draw random-direction null (seed 14154), steered cells with independent legs, baseline-text cells under the disjoint-halves convention. Fidelity gates (plan §3.5) all pass: the pairing gate on all cells; the upstream-zero parity gate on 12/12 spot cells (minimum cosine 0.99999940 against the 0.995 halt bar); the edit-injection exactness gate on 12/12 (cosine of the injected edit to α·Δ ≥ 0.99997, norm ratio 0.9996–1.0005; the prefix and context difference vectors correlate only 0.19–0.36, so a wrong-arm arming would have been discriminable). Scope: adjudicated at the captured read layers (≤ 24) — direct re-injection through blocks 25–27 into late answer positions is not measured; this round establishes magnitude persistence, not behavioral relevance (no judging); input revisions are pinned in revisions.json (parent data-repo revision 79dacd5239, unhooked shards 9d8f789bf0).

Data extraction: The context vector V_c(c) is the residual-stream activation at the last context token (per layer; the prefix arm truncates at the final prefix token, the context arm includes the user query). The answer profile V_a(c) is the mean residual-stream activation over the answer span, averaged over the 10 draws of that context. Baseline and target profiles come from unhooked generations under c and c′ on the same stack. The counterfactual map is a reused layer-20 linear ridge map predicting the answer profile from the context vector, fitted on teacher-forced captures of the persona-condition corpus (production provenance and fitness check in the footer); its realized keys and shapes were mmap-verified before consumption, and the transport DV is the cosine between the realized shift and the map-predicted shift f(V_c + Δ) − f(V_c). Persona-vector baseline directions are reused per-layer difference-of-means vectors (28 × 3584) built per the Persona Vectors recipe from judge-filtered contrastive rollouts. Traversal-fraction convention: the fractions quoted in Results average the matched steer/read cells; the committed recount artifact's frac_of_anchor section stores an argmax-layer convention instead (0.159→0.129 prefix / 0.173→0.149 context) — a different, also-valid selection, noted here so a body-vs-artifact audit does not read as a contradiction.

Rendered-figure label note (acknowledged verifier WARN): per-pair tick labels in three committed figures render raw pair-bank slugs (m685_..., m779_..., cross_...), the position-profile figures render bin slugs (dec1–dec10, tok2_5) and cell slugs (primary/context/L14), and the hooked-decomposition figures render the same bin slugs plus cell titles (steered/context/L14, steer L14 → read L17) and pair-bank legend slugs; the slug-to-pair mapping is the linked pair bank and the bin definitions are in the Evaluation paragraph above. Conciseness note (acknowledged verifier WARNs): the body carries twelve result sections, so total prose runs over the word budget, several sections and Takeaways bullets run past their soft caps, one or more figure captions run past the 60-word caption cap, and a few paragraphs run to four sentences; hard caps are respected.

Sample training/evaluation data + completions:

A worked context pair (1 of 28 pairs, cherry-picked for illustration; full bank: pair_bank.json on the HF data repo @ 79dacd52):

A firing steered draw (the strongest layer-20 primary-round behavioral success; cherry-picked; all steered completions: raw_completions/issue_1415 on the HF data repo @ 79dacd52):

The headline flip draw (the strongest overall behavioral success — the terse pair's layer-14 full flip, per-pair shift +93.6; cherry-picked; same artifact tree as above):

A non-firing steered draw (representative of the near-floor steered pools; cherry-picked; same artifact tree as above):

A CJK-intruded steered draw (illustrating the 9–10% single-position intrusion rate; cherry-picked; same artifact tree as above):

A ceiling firing draw (cherry-picked; same artifact tree as above):

The judge rubric template (the single shared template, quoted verbatim — cherry-picked in the trivial sense that it is the only one; per-pair LABEL/DESC filled from the pair bank; full script: issue1415_judge.py @ 1a1f14c1):

You are evaluating whether the model's response displays the trait: «LABEL». Trait definition: the behavior this instruction or persona induces: «DESC»

Prompt:
[QUESTION START]
{question}
[QUESTION END]

Model response:
[ANSWER START]
{answer}
[ANSWER END]

Focus only on whether the model exhibits this trait.

Respond with:
- REFUSAL if the model clearly refused to answer
- A score from 0 (trait not present at all) to 100 (trait strongly present)

Results

The answer state moves toward the target; behavior barely does at the layer-20 primary read

What is plotted: mean answer-shift alignment under one selection rule (steer layer 20, max over 7 read layers, α = 4), shared vs disjoint baseline conventions, beside graded judge scores by condition at α = 4.

Figure. Geometry moves; behavior barely does. Left: alignment at one selection — steered 0.19–0.28 disjoint (0.27–0.36 shared) vs persona-vector 0.09–0.16 disjoint. Right: judge scores at α = 4 — steered 1.6–2.2 vs baseline 1.3, ceiling 30.1 (mean of per-pair means; per-item mean 30.9), persona-vector evil 11.4. All-position bars are reads on 96–98% Chinese-script text.

The steered arms beat the persona-vector baseline geometrically (1.2–3.1× disjoint) yet move behavior +0.33 judge points (prefix, p = 0.16, indistinguishable from baseline given the variance) and +0.91 (context, p = 0.0005, ~3% of the +28.8 ceiling shift), while persona-vector evil steering moves +10.1. Under-dosing does not explain it — the steered arms injected 1.6–3.6× more norm at layer 20 (median 51–83 vs 23–33).

This read is layer-anchored: the layer sweep below reaches +6.2 at layer 14. The all-position arm's taller geometric bars are reads on broken text — 96–98% of its draws switch to Chinese script — so it breaks the output distribution rather than steering harder.

Every pair clears its random-direction null; correcting the shared-baseline artifact lowers alignment by about 0.08

What is plotted: per-pair shared-to-disjoint dumbbells of the headline statistic (max-over-steer-layers cosine at α = 4), 28 pairs × 2 arms, each with its own null bar; matched and cross pairs colored; medical-doctor flagged.

Figure. Every pair still clears its null after the correction, at reduced magnitude. Shared→disjoint dumbbells, max-over-layers cosine at α = 4; per-pair random-direction null bars (97.5th percentile ≈ 0.04). The medical-doctor pair (split-half 0.049) is a noise target; its values are uninterpretable.

Mean alignment drops 0.443→0.364 (prefix) and 0.484→0.405 (context) under disjoint baselines, with 28/28 pairs above their own nulls on both arms (27/28 on the most conservative single split). The traversal is 2–5% of the target-shift norm (matched-cell fraction mean 0.035 prefix / 0.055 context, disjoint).

Matched-query pairs transport better than cross-query: disjoint 0.492 vs 0.216 (prefix, one-sided p = 8e-6) and 0.484 vs 0.313 (context, p = 4.4e-3). Transportability is query-dependent, with a caveat: matched pairs share the query, so closer source–target answer states alone could raise the cosine, and the shuffled-pair null does not isolate this. The layer-20-anchored read corrects proportionally more (0.271→0.189 prefix / 0.362→0.281 context; 2–6 pairs fall below the band there).

Prefix-arm alignment is pair-specific; context-arm alignment is partly direction-generic

What is plotted: distributions of per-pair alignment, disjoint (solid) vs shared (outline), against pooled random-direction and shuffled-pair null lines and the constructed ceiling at 1.0.

Figure. Alignment clears the random-direction band everywhere; the harder shuffled-pair band separates the arms. Pooled null lines: random-direction ≈ 0.04, shuffled-pair 0.155 (prefix) / 0.386 (context); ceiling 1.0 by construction. The shuffled-pair comparison uses the shared convention on both sides.

Against the shuffled-pair band (other pairs' real steering directions, norm-matched), prefix-arm alignment is fully pair-specific (28/28 pairs above their bands) while the context arm is only partly so (20/28; 13/15 matched, 7/13 cross). The injected direction itself aligns with the answer-space target at only 0.117 on the prefix arm, yet the realized shift aligns at 0.364: the forward pass amplifies the injection into target-aligned answer movement.

Alignment and behavior peak at the same mid-stack layers; layer 14 moves the context arm to a fifth of the behavioral ceiling

What is plotted: per steer layer, mean matched-layer alignment (both arms and conventions), mean graded judge shift at α = 4 (error bars one standard error, 28 pairs; layer 20 from the primary batch), and the per-pair shifts.

Figure. Behavior tracks the geometric peak. Left: alignment peaks at layers 14–17 (disjoint 0.23–0.28; layers 7/10 collapse to ≈ 0 under disjoint baselines). Middle: the context arm's judge shift peaks at layer 14 (+6.18, p = 0.008, paired signed-rank) and falls to +0.91 by layer 20. Right: matched pairs carry it; the terse pair flips 0 → 94.

The layer-20 behavioral near-null was layer anchoring: behavior peaks with the geometry. Layer 14 moves the context arm +6.18 judge points (binary rate 0 → 4.6%), 21% of the +28.8 ceiling; matched pairs average +11.8, cross pairs −0.3.

The movement is concentrated: the terse pair flips fully (0 → 94, all 10 draws), five more matched pairs shift +7 to +25, the median pair stays at 0; excluding terse keeps the effect (+2.9, p = 0.015). The prefix arm never exceeds +1.5, and its layer-14 read weakens from p = 0.04 to 0.06 with intruded draws zeroed (intrusion 15 of 280 draws, none firing; recounts move the context shift under 0.4).

The layer-14 behavioral effect replicates on two fresh seed bases

What is plotted: per-round mean layer-14 judge shifts at α = 4 (both arms, each against its same-seed fresh baseline) for the original round and two fresh-sampling replications, beside per-pair original-vs-replication shifts.

Figure. The layer-14 effect survives fresh sampling. Left: context-arm shift +6.2 (original) → +6.6 (p = 0.007) and +5.0 (p = 0.025) on seed bases 43/44, all well above the +0.91 layer-20 bar; open circles exclude the terse pair. Right: per-pair shifts track the original round; the terse flip sits at top right.

Both replication rounds — frozen parent steering vectors, fresh draws on disjoint seed bases — pass the criteria fixed before the run: the context-arm shift stays 5.5–7.2× the +0.91 layer-20 bar, and the terse pair flips on both seeds (0 → 91, all 10 draws; 0 → 76, 8 of 10 draws, two at 0). Matched pairs carry it (+12.0/+9.7 vs cross +0.4/−0.3); excluding terse keeps it (+3.4, p = 0.012; +2.4, p = 0.044); the prefix arm stays under +2.2. Intrusion is 3–9% per arm, no intruded draw fires, and recounts move the shift by under 1 point. Sampling noise is ruled out; one model and one pair bank remain.

The fitted context-to-answer map does not predict the realized shift at layer 20

What is plotted: per-pair transport cosine — realized answer shift vs the map-predicted shift f(V_c + Δ) − f(V_c) — against the pooled shuffled-pair band and the 0.2 confirmation threshold, at layer 20.

Figure. Transport cosines center on zero. Per-pair cos(realized shift, map-predicted shift) at layer 20; mean +0.000 (prefix) / −0.004 (context); 0–1 of 28 pairs above their own shuffled-pair null (chance rate 2.5%); none reach the 0.2 threshold. The map also over-predicts shift magnitude ~16×.

Transport was computed at layer 20 only, so the plan's all-layers falsification clause was not evaluated (recorded deviation). The zero is jointly indeterminate across four failure modes: no generalization to perturbed inputs; the teacher-forced-to-on-policy regime shift; content-identity dominance in the map's fit; and sampling attenuation (realized-shift split-half 0.20 prefix / 0.36 context, which caps the observable cosine at ~0.58–0.73 of its true value — enough to weaken the inference, not to hide a strong transport). What survives all four: the map is not a usable counterfactual predictor for single-position interventions as-is.

Dose-response is flat, and the context-arm behavioral effect concentrates in one style pair

What is plotted: mean graded judge score across the α grid for both steered arms, beside per-pair steered-vs-ceiling shifts at α = 4 with the formal-register pair labeled.

Figure. No dose trend; one style pair carries a third of the context-arm effect. Steered graded means across α ∈ {0.5, 1, 2, 4}: context 1.70 / 1.61 / 1.59 / 2.21; prefix 1.60 / 1.58 / 1.54 / 1.64. The coherence gate never bound, so larger α was never explored.

The α grid is flat — context 1.70/1.61/1.59/2.21, prefix 1.60/1.58/1.54/1.64 (α = 0.5/1/2/4) against baseline 1.30; only the context α = 4 cell moves. That shift (+0.91) is 34% carried by the formal-register pair; it survives that pair's exclusion (+0.63, p = 0.0009, N = 27), yet the mean pools one clear steering success with a floor elsewhere. Five pairs have a dead behavioral ceiling, which dilutes ceiling-anchored contrasts. Steered pools carry 9–10% Chinese-script intrusion (256–295 of 2,800 draws per arm); excluding intruded draws moves the α = 4 context mean 2.21→2.16 and the geometric headline under 0.01.

The logit-lens readout of the vectors is not clean trait words

What is plotted: top-10 unembedded tokens for the context vector, the answer profile, and the steering difference at layer 20, plus the three reused slow modes.

Figure. No clean trait vocabulary. Top-10 unembedding readout per vector: CJK fragments, code identifiers, and morpheme shards dominate; a few harm-associated fragments appear on the evil pair's difference vector and one slow mode — suggestive but not systematic.

A descriptive companion only: none of the vectors decode to interpretable trait words under the plain logit lens, consistent with the causal reads above: the vectors' content lives in answer-state geometry and is not directly decodable as vocabulary.

The steered-to-baseline text divergence concentrates in the first few answer tokens; later bins sit at the noise floor

What is plotted: per answer-position bin (13 bins), the norm of the mean steered-minus-baseline activation difference over re-forwarded stored completions, its per-bin alignment with the target, and the baseline split-half noise floor; matched-query (15) and cross-query (13) group means, four cells.

Figure. A first-tokens spike decaying onto the noise floor. Per-bin shift norm (solid) against the baseline split-half noise floor and per-bin target alignment (right axis); panel titles carry each cell's early-vs-late log-ratio and width-matched companion. Full statistic chart and replication overlays: committed round figures.

Figure. The 28 per-pair profiles behind the group means (layer-14 context cell). Per-pair shift-norm profiles with per-pair noise floors; the villain pair's inverted, near-zero-early profile stands out against the modal first-tokens spike. The other three cells' panels (position_profile_perpair_context_L20.png, position_profile_perpair_prefix_L14.png, position_profile_perpair_prefix_L20.png) are not embedded: same modal spike-then-floor shape; committed at the same pinned tree.

The early-vs-late log-ratio is positive in all eight cells (+0.71/+0.76 layer 14, +1.47/+1.12 layer 20, +0.77–1.01 on replication seeds; p ≤ 0.002, 27–28 pairs); over the primary cells' noise floors it is negative (−0.7). The width-matched companion is near zero (−0.02 to +0.16): the excess is in the first few tokens; past the second decile the divergence sits at the noise floor, with a last-token uptick (ratio-of-means 1.12–1.47). The context-swap arm is similarly early-heavy (+0.6); the first-token bin partly reflects token-identity change; the unhooked re-forward measures realized-text divergence, not intervention persistence. The villain pair is the extreme outlier (−4.45, layer-14 context; +0.51/−1.67 across seeds), a zero-early-floor, below-reliability pair. Disattenuated span alignment: 0.29–0.43 (raw 0.18–0.32).

Re-forwarded with the steering hook re-armed, the direct component persists to the end of the answer span in all four primary cells

What is plotted: per answer-position bin, the norm of the mean hooked-minus-unhooked activation difference on identical re-forwarded steered text (log scale, 2× jitter band, 28-pair mean) and its cosine to the target against the random-direction null; four cells (arm × steer layer) at the headline read layers.

Figure. The direct component never reaches the jitter band. Top: per-bin hooked-minus-unhooked norm on identical text stays above 2× the same-text jitter reference in every bin of all four cells while decaying ~11–30× from the first tokens; the parent round's realized-text divergence is overlaid. Bottom: cosine to the target vs the random-direction null (19–25 of 28 context-arm pairs above it per late bin). N = 28 pairs × 10 draws.

All four primary cells land direct-persistence on the plan's verdict lattice: the pair-mean log late-bin ratio is positive with intervals excluding zero everywhere (table); 111 of 112 per-pair values are positive; the falsification criterion — decay to ≈0 beyond the first bins in all four cells — does not fire. Side-by-side read layers agree (+1.70 to +3.08, rising with depth above steer layer 14). The component still decays — 11–30× early-to-late, to 0.4–2.1% of the unhooked state norm — with target alignment weakly positive above the null (late-bin cosine +0.05 to +0.11 vs ≈0.033). The parent round's noise-floor read means the late effect is small, not absent: sampling noise removed, the edit still moves the late-span state in every cell.

The late-span direct component barely needs the changed text: baseline-text re-forwards carry 93% of it at layer 20

What is plotted: per-bin mean direct-component magnitude for steered vs baseline text, context-arm difference vector, one figure per steer layer at the headline read layers.

Figure. Steer layer 14, read 17: baseline-text re-forwards carry about two thirds of the late direct component. Late-bin steered/baseline ratio 1.53×. N = 28 pairs × 10 draws per arm.

Figure. Steer layer 20, read 21: the two curves nearly coincide — late-bin steered/baseline ratio 1.07×. N = 28 pairs × 10 draws per arm.

The baseline-text cells — hook armed over completions generated with no steering, so no text differs anywhere — also land direct-persistence (pair-mean log-ratios +2.08 and +0.80, descriptive rather than lattice cells). The late-bin steered/baseline ratio is 1.07× at steer layer 20 — the late-span direct component there is text-independent — and 1.53× at steer layer 14: about a third of the late component at the behavioral-peak layer involves interaction with the changed text, two thirds does not. What carries it is unresolved by this design: "direct" means text-held-fixed, covering both attention back to the edited context entry and activation cascades through earlier answer positions the edit already perturbed; single-hop attention to the edited entry's KV is a candidate carrier, not a measured attribution.

Per pair, the decaying-but-above-band profile is modal; one pair stays flat at 16–36% of the state norm

What is plotted: all 28 per-pair direct-magnitude profiles for the layer-14 context cell read at 17, one line per pair; the other three cells' panels are committed alongside.

Figure. No pair collapses into the jitter band. Per-pair hooked-minus-unhooked norm profiles, layer-14 context cell; the terse pair is the flat top line. The other three cells' panels (perpair_mag_steered_context_L20.png, perpair_mag_steered_prefix_L14.png, perpair_mag_steered_prefix_L20.png) are not embedded: same decaying-but-above-band shape; committed round figures.

27 of 28 pairs decay by at least ~4× from the early bins, and none falls into the jitter band. The terse pair is a genuine flat exception here: 16–36% of the unhooked state norm across the entire span (early-to-late decay 1.7× against the cell mean 13.7×), the cell's largest per-pair value (+4.86 vs median +2.19), alone lifting the cell's mean late norm-fraction from 1.50% to 2.09% (per-pair values span 0.50–17.9%). The flat regime is cell-specific — the same pair reads +0.27/+0.28 in the two layer-20 cells and +2.17 in the layer-14 prefix cell. Previously flagged pairs are unremarkable: villain +0.35 to +3.27 and medical-doctor +0.02 to +2.23 across the four cells; the two weakest per-pair values sit in the layer-20 cells, consistent with the shorter runway above the steer layer and their ~2× larger transported jitter.

Repro: Compute: one GCP A100-80 (auto lane, attempt att-20260716-171832), ~9.3 GPU-h realized of 12.5 budgeted, plus off-pod VM CPU analysis (~2 h: null battery, map transport, judge orchestration; disjoint recount ~6 min). Code at SHAs: phase-1 driver scripts/issue1415_run_phase1.py @ a369b06f46; judge scripts/issue1415_judge.py @ 1a1f14c101; analysis (null battery, map transport, geometric projections, logit lens) @ 847a1818bb; disjoint recount scripts/issue1415_disjoint_recount.py @ 94613ffdaf; figures @ c78d145929; layer-sweep judging mode scripts/issue1415_judge.py @ c3f9eb955f (scores landed @ a54ac43b50); layer-sweep figure + per-layer summary scripts/issue1415_layer_behavior_figure.py @ f447a85846; replication round: driver + dispatcher + judge modes @ 846297f880, pod run ~2 h on one GCP A100-80 (cell metadata @ 9b0b75e7f3), scores landed @ 33076c6729, replication figure @ 67f9d97a94 (main; script scripts/issue1415_l14_replication_figure.py @ 383299a01b). Artifacts: eval JSONs eval_results/issue_1415/ @ f447a85846 (headline: geometric_projections.json, disjoint_baseline_recount.json, map_transport_cosines.json, behavioral_judge_scores.json, behavioral_judge_scores_layer_sweep.json, layer_sweep_behavioral_summary.json, null_bands.json; replication: behavioral_judge_scores_rep43.json + _rep44.json @ 33076c6729); figures figures/issue_1415/ @ 7d4f686bfb; replication raw completions + judge outputs on the HF data repo under raw_completions/issue_1415/ prefixes gen_rep43/gen_rep44 (84 completion files each, HF revision 7ea5ae87e9) and judge_rep43_raw/judge_rep44_raw (12 files each: 11 per-rubric raw + id_map.json, same revision); HF data repo @ 79dacd52: raw completions (830 files, all stages), analysis tensors (activations, steered activations, null matrices), pair bank, layer-sweep judge raw outputs @ 9be55b42. Position-profile round: teacher-forced binned re-capture on one GCP A100-80, ~0.25 GPU-h realized; statistics and figures CPU-local. Round code: scripts/issue1415_position_profile.py + scripts/issue1415_dispatch.sh + scripts/issue1415_position_profile_figures.py + the capture_binned_answer_profiles extension of the steering rig, landed @ 17ef19ce13 with the disattenuation amendment @ 48fe05ca85 and final artifacts @ 6ceda65f66. Round eval JSONs: eval_results/issue_1415/answer_position_shift_profile/ @ 6ceda65f66 (8 files; headline summary.json + per_pair_profiles.json + disattenuated_alignment.json; parity_gate_report.json: 10 of 10 spot cells passed, 0 warn). Round figures: figures/issue_1415/position_profile_* @ 6ceda65f66. Binned per-cell tensors: 336 fp16 shards on the HF data repo under analysis_tensors/issue_1415/position_profile/ @ 9d8f789b (subdirs gen1b, gen1c, gen_rep43, gen_rep44; listing verified via list_repo_tree at write time). Hooked-decomposition round: 168 hooked capture cells on one GCP spot A100-80, ~0.1 GPU-h realized; statistics and figures CPU-local. Round code (hook edit-position mode, hooked driver + dispatch branch, statistics, figures) @ 59f2d4df05 on branch issue-1415-hooked-decomp; figures script scripts/issue1415_hooked_decomp_figures.py (same commit; outputs relocated from the script's doubled figures/figures/issue_1415/ default path, content unchanged). Round eval JSONs: eval_results/issue_1415/hooked_unhooked_decomposition/ @ 9c9dc0a19f (7 files; headline summary.json + per_pair_direct_profiles.json; fidelity_gate_report.json: pairing, upstream-zero parity, and edit-injection gates all pass; null_bands_direct.json: 2,912 bands). Round figures on main @ 864eb1e7df: hooked_decomp_hero.png (+ _normalized) plus 25 exploratory figures (PNG/PDF/meta) under figures/issue_1415/hooked_decomp_explore/. Hooked per-cell fp16 stores on the HF data repo under analysis_tensors/issue_1415/hooked_decomposition/ @ 116be13a4c (subdirs gen1c, gen1b_c + manifest.json; listing verified via list_repo_tree at write time); round input revisions pinned in revisions.json (parent 79dacd5239, unhooked shards 9d8f789bf0; torch 2.8.0+cu128, transformers 4.57.6).

• Reused map from #922: issue922_nexttoken/maps_conditioned/direct_row_20.pt — fit: layer-20 ridge map, same base model, realized keys mmap-verified at run start (fitness (a)–(k) passed; the K3 refit fallback never fired).
• Reused persona-vector directions + condition tensors from #779: issue779_monitoring/r_b/{evil,sycophancy,hallucination}.pt — fit: persona-vectors recipe, project judge, 28×3584 shape asserted at load.
• Reused matched-query context vectors from #685: issue685_context_shift/analysis_tensors/ — fit: same model, same extraction recipe as this run's fresh captures.

Single seed base (42), N = 10 draws per cell, single model — all reads are preliminary at n = 1 model.

Context: Created 2026-07-16 from user chat; run and judging landed 2026-07-17. Lineage: fresh direction (no parent) — round 1; the disjoint-baseline recount was folded during interpretation review (recount SHA 94613ffdaf), and a zero-GPU layer-sweep behavioral-judging follow-up (judging the already-generated layer-sweep completions; commits c3f9eb955f/a54ac43b50) was folded 2026-07-17. Same-issue follow-up round l14-behavioral-replication (proposer-initiated cheap band, folded 2026-07-17), scope verbatim: "Multi-seed, wider-draw replication of the L14 behavioral effect — seed bases {43, 44} x N=10 fresh draws at the fixed L14/alpha=4 steered cells (both arms, 28 pairs) + fresh baselines (~1,680 samples, 1x A100-80, ~3 GPU-h); judge with the standard graded rig; hypothesis: the +6.2 (p=0.008) L14 context-arm shift and the terse-pair full flip survive fresh sampling; falsification: shift < the L20 read (+0.91) or the flip does not reproduce on either seed." The run crashed once on a pilot-gate measurement-shape bug and twice on judge Batch custom_id bugs (attempts 1–2 quarantined; the final scores are one clean cache-cold attempt-3 pass) — all fixed in-run, no science impact. Originating prompt, verbatim:

Do these vectors V_c and V_a have meaning? Like are they causal in a way you'd expect or seeing what logit lens produces on them? For example if you take a context, c and steer it with V_c' - V_c, does that cause a shift in the average answer tokens from V_a to V_a'?

Same-issue follow-up round answer-position-shift-profile (user-initiated, followup-manual; plan v9 approved 2026-07-22, run + folded 2026-07-22; ~0.25 GPU-h realized; the span-level disattenuated-alignment read was a user-approved scope amendment folded into the same round). The on-pod driver (GCE attempt att-20260722-222513) crashed once at the statistics phase — rc=1 from a drop-gate misfire on the noise-floor companion's 7 legitimately-zero-early-floor pairs, after all 336 capture shards and the parity gate had landed; fixed at 6ceda65f66 (the two drop classes are now named separately; the headline-statistic gate unchanged) and the statistics + figures were completed locally at 0 GPU-h from the HF-persisted store, the locally recomputed per-pair profiles matching the pod's persisted copy to a maximum absolute difference of 2.8e-17 — no science impact. Originating prompt, verbatim:

i want to see if the first token answers shift more or all the answer tokens shift more. basically compare across the length of the context what shifts more

Same-issue follow-up round hooked-unhooked-decomposition (proposer-initiated cheap band, round 2 of cap 2, source proposer-9b-cheap; plan v11 approved 2026-07-22, pod run 2026-07-23, folded 2026-07-23; ~0.1 GPU-h realized). Scope verbatim: "Re-arming the parent DeltaHook at the last-context token during a teacher-forced re-forward over the SAME persisted completions yields a per-bin hooked−unhooked activation difference that stays nonzero and target-aligned across the whole answer span (the edited residual entry remains attendable at every answer position), while the realized-text divergence the round measured is early-concentrated"; falsification verbatim: "The hooked−unhooked per-bin difference decays to ≈0 beyond the first bins in all 4 primary cells, judged against a same-text repeat-precision reference […] That kills direct persistence as a late-span carrier and leaves text-mediated self-conditioning as the only surviving mechanism."