Methodology: docs/methodology/issue_825.md · gist

Takeaways

• Pretrained Qwen2.5-7B carries the linear context→answer-profile map at 87.3% of instruct strength (held-out R² 0.588 vs 0.673, layer 19); representation-swap and map-swap reads are consistent with a rotated read-out.
• An exogenous separator→span control transfers only 5.7% (base) and 10.9% (instruct) of the chat map's information; the control map is model-shared (weight cosine 0.86–0.90), unlike the rotated chat map.
• Own-continuation controls close D 0.590 (base, MLP-carried) and 0.428 (instruct) of the exogenous-to-ceiling gap — the registered partial band; an unexcluded instruct position residual would naively lower D to ≈0.30.
• The assistant map beats the user map in all 24 pair-layer reads (ridge R² gaps +0.36 to +1.53; paired CIs clear of zero) — a role-slot-bundle contrast, not mechanism evidence.
• The user-turn linear null holds under all three user-text provenances (ridge R² −0.92 to −1.12 on 2000 real conversations); user and two-turn assistant maps are nonlinearly recoverable (MLP 0.17–0.19, 0.49–0.56).
• Real logged conversations depress the assistant map (MLP 0.32–0.38 vs 0.49–0.56); self-written user turns beat Haiku-written in three of four cells; every rig gate across all seven rounds PASSed.

Goal

This experiment in context: The parent experiment (#779) established a strong linear map from a context's mean residual-stream activation to the model's answer profile on single-turn Qwen2.5-7B-Instruct, peaking near layer 19. This experiment holds that rig fixed and asks two questions: does the same linear map already exist in the pretrained base model (before any post-training), and does it hold when the prediction target is the user's next turn rather than the assistant's — extended with a two-turn conversational track, cross-role transfer cells, a user-turn-provenance axis (Haiku-written vs self-written vs real logged human user turns), a direct base↔instruct map comparison, and a formally paired within-conversation role contrast across all three provenances.

Broader narrative: Whether the context→answer-profile map is inherited from pretraining or created by post-training locates where context/persona conditioning enters the training pipeline; whether it is assistant-generation-specific or a general next-turn property bounds how far base-model context geometry can be used to predict downstream fine-tuning behavior (the project's leakage-prediction line).

Methodology

Rounds: Round 1 = the parent run (below). Round 2 (mlp-unprobed-cells, cheap-band auto-run): the parent's exact MLP probe extended to the 6 unprobed cells via scripts/issue825_mlp_followup_dispatch.sh @07589e890b — turnstore staged from HF @deb7a452 (4 m-track prefixes), 8-cell ridge refit as carrier + two-anchor ±0.05 regression gate (PASS, deltas +0.0000), 6-cell MLP, no recipe changes; outputs at eval_results/issue_825/mlp-unprobed-cells/. Three launches (~2.5 GPU-h vs 2 budgeted): two stage hangs (xet finalization, fixed by HF_XET_DISABLE=1), then a fully-successful fit whose gate crashed on a wrong dict path — outputs rescued from the crash-persist upload, corrected gate re-run locally. Round 3 (onpolicy-user-turn, user-requested): one changed variable — second-user-turn provenance, Haiku-written → written by the measured model itself per (model, format) cell; parent fit recipe unchanged; 4 assistant-map anchors refit as a rig gate; matched-rows parent refits for the two cells whose keep rate fell below 95%; via scripts/issue825_onpolicy_dispatch.sh @debc28c31d, outputs at eval_results/issue_825/onpolicy-user-turn/. Four GCP launches (~2.6 GPU-h vs 3 budgeted): three crashed in extraction on a zero-width-span tokenizer edge case in short self-written turns (fixed by generation-time span validation), the fourth ran clean. Round 4 (real-user-turn-null, cheap-band auto-run round 2): second-user-turn provenance changed to real logged human turns — 2,000 fully-logged 2-turn lmsys-chat-1m conversations (u1, a1, u2 all logged; a1 written by the logged serving models) ingested at the parent's pinned revision, no generation anywhere; 8 within-cell fits (4 user + 4 same-conversation assistant references) + 1 end-to-end parent-anchor re-extraction, parent fit recipe unchanged, via scripts/issue825_realuser_dispatch.sh @9e821f906f; outputs at eval_results/issue_825/real-user-turn-null/. Five launch attempts (~2.7 GPU-h vs 3 budgeted): one GCP flex-start instance preempted three times mid-extraction by the provider, failed over to RunPod, one wiring-phase crash on a Unicode line-separator JSONL edge case (fixed), then a clean run. Inline free analysis (crossmodel-map-transfer, user-requested, 0 GPU-h): cross-model map-swap / representation-swap + weight-space similarity of the two fitted maps on the existing turnstore, scripts/issue825_crossmodel_map_transfer.py @4d03165dd8 (pin corrected 2026-07-07: the file was created in 4d03165dd8; the earlier 5b5b53bbe2 citation predates its existence), 100 permutation-null draws, ridge core reused verbatim from issue825_fit_cells.py. Round 5 (role-map-comparison, user-requested): one changed variable — the ESTIMAND, from per-cell absolute R² to the paired within-conversation role contrast (assistant − user) — with zero new data: a tensors-only refit of all 24 within-cells from the three persisted turnstores (12 bundles + on-policy row allowlists, 172.5 GB staged at data-repo revision a23e79f17f), one joint keep mask per pair so both roles share rows and fold assignments by construction, paired conversation-level bootstrap (1000 shared resamples, per-draw own-mean re-centered, fp64), paired MLP fold deltas at layers 19 and 26, per-example cosine and NLL companions, and a 20-cell reproduction gate (±0.01) against the committed values; via scripts/issue825_rolecontrast_dispatch.sh + scripts/issue825_role_contrast.py @e8fe5ca633, outputs at eval_results/issue_825/role-map-comparison/{haiku,onpolicy,real}/. One GCP flex-start launch, clean run, 1.465 GPU-h measured (4 budgeted); no plan deviations. Round 6 (base-separator-control, user-requested): one changed variable — the model under the separator/punctuation specificity control (the story-map control recipe; lineage in the footer), Instruct → pretrained base; corpus + pair construction + estimator ladder + transfer protocol reused verbatim at pinned SHAs (pair files @ HF rev 9534b998: 3,600 pairs / 600 WikiText-103 articles, sentence-final anchors {., !, ?}, spans 8–256 tokens, plus the preceding-sentence variant; extraction issue931_extract_store.py @a4989992 bf16 batch 8 + two-bar equivalence gate (committed read 0.99995 / 0.99979, armC_equivalence.json on the HF store prefix), extended with a default-preserving --model-id flag; GCV Gram ridge λ ∈ logspace(−2, 4, 13), 5 article-group folds seed 0, 20 group-blocked shuffle nulls, 1,000-draw group bootstrap; rotated random-projection control; batched MLP secondary at frozen layers {14, 18, 19, 26}); two wiring gates before any decision read — instruct-anchor refit ±0.01 (PASS: ridge −3.168/−3.497, rotated +0.349/+0.334 reproduced) and the chat-stream validation gate (base chat within refit reproduced the committed 0.5877 exactly); recentered sep→chat transfer at layer 19 vs the base chat ceiling (full-n + matched-1982, subsample scheme issue931_pcms.seeded_uniform_row_draw.v1 seed 931, single-draw AND 5-draw conventions recorded) + base-vs-instruct map-swap / representation-swap / weight-space reads reusing issue825_crossmodel_map_transfer.py cores; new scripts issue825_base_sep_{dispatch.sh,stage.py,transfer.py,similarity.py,figures.py} @1c284be781; outputs at eval_results/issue_825/base-separator-control/. Two spot-A100-80 launches (~2.5 GPU-h vs 2 budgeted, ≤3 scope cap): the first preempted by the provider mid-fits, the second clean. The control's raw-text spans carry no user-query split, so the project's prefix/context dual-arm convention is N/A for this arm (single-anchor recipe follows the parent control recipe verbatim — stated deviation). Round 7 (onpolicy-separator-control, user-requested): one changed variable — the span-text SOURCE of the round-6 separator control, exogenous WikiText article text → each model's own greedy raw-text continuations of prefixes of the SAME 600 pinned articles (no chat template; vLLM greedy seed 42; 768-token wave-1 continuations from 256-token prefixes plus a 512-token wave-2 top-up from 512-token prefixes on low-yield articles; new scripts/issue825_onpolicy_sep_gen.py reusing round-3's engine guards and audit helpers). Pair construction reuses the story-map control pair recipe verbatim (same helpers and filters as round 6; lineage in the footer) except anchors are restricted to the continuation region (scripts/issue825_onpolicy_sep_pairs.py, consumer-exact filenames); extraction, fit ladder, anchor staging, and the recentered transfer protocol are reused at pinned SHAs with two default-preserving source extensions (a flag-gated 1,000-draw group-bootstrap CI over the ROTATED estimator's held-out predictions in the fit script, and source-side --source-store-dir/--model flags on the transfer script). Decision reads registered in plan v21 §1: per model, the position statistic D = (W_on − W_ex) / (C − W_ex) under the round-6 max(rotated, MLP) convention against frozen committed references (base W_ex 0.36261 / ceiling C 0.58768; instruct 0.34892 / 0.67309), tertile bands 0.33/0.67 with CI-qualified edges; the recentered on-policy separator→chat transfer at layer 19 as a fraction of the full-n chat ceiling vs the 0.5 specificity line (matched-n denominator documented unevaluable on base); a ±0.10 cross-substrate mirror margin with the raw gain ΔW reported next to D (the D denominators differ, 0.22507 vs 0.32417); and a matched-n W_ex re-baseline trigger at realized n < 3,492 (not fired: realized 3,577/3,591). Anchor gates extended to BOTH models' exogenous cells (±0.01). Via scripts/issue825_onpolicy_sep_dispatch.sh @b3a428d1c0 (plan v21), outputs at eval_results/issue_825/onpolicy-separator-control/; one GCE flex-start A100-80 launch, clean run, 2.907 GPU-h measured (3 budgeted); the VM Phase C (transfer + figures) was relaunched once after an output-routing fix (@fda8f98ba6). Round-7 revision pass (interp-critique round, 0 GPU-h, VM CPU): the plan-registered position-matched exogenous refit — exogenous pairs restricted to anchor token position ≥ 256 (meta.anchor_pos in the pinned pairs @9534b998; 2,701 of 3,600 pairs kept, all 600 article groups), rotated-estimator refit at the frozen layers with the committed machinery (identical random-projection draw stream, group 5-fold cached-eigh GCV ridge, 20 group-blocked pairing-shuffle nulls at layer 19), plus a size-matched position-agnostic group-stratified random-subsample control (3 seeds) separating the position restriction from the n drop; the full-n refit reproduces both committed anchor rotated values to machine precision (deltas 0.0); scripts/issue825_position_matched_wex.py @5db8b48bd4, outputs position_matched_wex_{base,instruct}.json. A second revision pass (interp-critique round 2, 0 GPU-h, VM CPU) extended the instruct subsample control to 10 seeds via the same script's --extend-subsample mode against the staged stores (validation: seeds 1–3 reproduce the 3-seed values to machine precision), sidecar position_matched_wex_instruct_v2.json @ad9a2f047a.

Design: Two models (pretrained Qwen2.5-7B; Qwen2.5-7B-Instruct) × two target roles (assistant, user) × two context formats (chat template, naturalistic transcript). The single-turn track (Track S; n = 5000 prompts per cell, 2 cells) replicates the parent rig on both models; the two-turn track (Track M; n = 2000 conversations per cell) fits 8 within-cell maps plus 4 cross-role transfer cells. Round 3 adds a user-turn-provenance arm: the same 4 user cells refit on conversations whose second user turn each measured model wrote itself. Round 4 completes the provenance axis with real conversations: the full 8-cell grid refit on 2,000 fully-logged human 2-turn conversations (both user turns human-written; first answers from the logged serving models), plus 1 parent anchor. Round 5 adds no data: it refits both roles of every conversation on shared folds to make the role contrast itself the estimand — 12 pairs (3 user-turn provenances × 2 models × 2 formats), each pair one conversation set carrying both an assistant-turn and a user-turn target. Rounds 6–7 add a separator/punctuation control arm outside the conversation grid: per model, ~3,600 pinned WikiText pairs mapping a sentence-final separator token's activation to the following-span mean — round 6 on the original human-written article text, round 7 on the model's own greedy continuations of the same articles' prefixes (2 models × the sentence-separator cell + a preceding-sentence variant, plus per-model exogenous anchor refits). Single seed throughout (fit seed 0, generation seed 42).

Training: N/A — no model training. Complete analysis + generation constants:

Parameter	Value	Source
Ridge fit	closed-form ridge, generalized-cross-validation-selected λ, K = 5 folds, fit seed 0	#779 fit_h
Frozen layer set	14, 18, 19, 26 (headline read at 19)	#779 / #722
Shuffle nulls	selection-symmetric, 20 draws (ridge), 5 draws (MLP)	#778 / #779
Bootstrap CIs	n_boot = 1000	plan §11
Track-S sampling	1 sample/prompt, T = 1.0, top_p = 0.95, max_tokens = 1024, seed 42, chat template	#779 pass-B parity
Track-M assistant turns	vLLM greedy, seed 42	task body
Track-M second user turn	claude-haiku-4-5, T = 1.0, 22 rotated user-persona briefs	plan §11 (tier-3 synthetic)
Round-3 second user turn	written by the measured model itself (vLLM, T = 1.0, top_p = 1.0, max_tokens = 512, seed 42); chat cells: chat-template render + opened user header, stop <|im_end|>; naturalistic cells: transcript + User: , stop on blank line / Assistant:	plan v7 §11 (parent u2 sampling parity)
Round-3 row handling	user-cell filters applied at fit time via per-cell row allowlists (kept n = 1914 / 1722 / 1999 / 1738); anchors fit all 2000 rows	plan v7 (anchor row-parity fix)
Round-3 support margins	MLP: point delta ≥ +0.10 clearing 2 standard errors; ridge: delta ≥ +0.30 at layer 19 or 26, or any frozen-layer R² > 0	plan v7 §11
Round-3 audit floors	keep rate ≥ 0.80; distinct-3-gram rate ≥ 0.5 (headline inclusion, non-halting)	plan v7 §11 (80% yield convention)
Round-4 conversations	fully-logged real 2-turn lmsys conversations at the parent's pinned revision; strict user/assistant/user prefix, English, non-redacted, non-moderation-flagged, kept n = 2000 of 30,311 streamed; no u2 length cap beyond the 2048-token conversation filter; all 8 cells share the identical kept-2000 rows	plan v11 §2/§4
Round-4 cross-provenance band	MLP descriptive labels at point delta ≥ ±0.10 clearing 2 standard errors (this round's fold SE × √2 proxy, applied uniformly; parent folds not persisted — round-3 folds are persisted and drawn in the three-provenance figure); no support/kill labels on the MLP	plan v11 §1
Round-4 wiring check	own-context vs derangement-shuffled-context teacher-forced NLL, n = 200 rows per (model, format)	plan v11 §7
Row filters	≥ 8 content tokens, ≤ 2048 total tokens	plan §11
MLP probe	input = full-dimension standardized activations; target reduced to its top-64 PCA components (basis fit on train), predicted jointly, then un-PCA'd; 1 hidden layer of 512 GELU units, AdamW lr 1e-3, weight decay 1e-4, ≤ 300 epochs with early stopping (validation fraction 0.1, patience 20); 1800 s/cell CPU budget	fit_h.mlp_fit_predict @e011a0b1ca
Round-7 continuation generation	vLLM greedy (temperature 0), seed 42; max_tokens 768 (wave 1, 256-token prefixes) / 512 (wave 2, 512-token prefixes); max_model_len 4096; no chat template	plan v21 §11 (chat-side greedy convention; the control recipe's 1,024-token window cap)
Round-7 decision conventions	D tertile bands 0.33 / 0.67; mirror margin ±0.10; transfer threshold 0.5 (full-n); matched-n re-baseline trigger at realized n < 0.97 × 3,600; group-bootstrap CI over the rotated estimator, 1,000 draws	plan v21 §1/§11 (round-6 committed conventions)
Per-position store cap	≤64 positions/turn, peak layers only	plan §11 (storage bound; pilot ≥80% coverage)
n per cell	5000 (Track S) / 2000 (Track M)	#779 / plan §11
Models	Qwen2.5-7B (pretrained) vs Qwen2.5-7B-Instruct	plan
Activation dtype	bf16 (plan deviation from fp16: fp16 max 65504 overflows Qwen residual outlier dims)	run log

Evaluation: The dependent variable is held-out R² (5-fold cross-validation) of a ridge map h from c_x — the mean residual-stream activation over the context tokens at a given layer — to v(x), the answer-profile vector of the target turn; R² below 0 means the map predicts worse than the global mean. Teacher-forced NLL of the realized target turn is the secondary read. Two read modes: FROZEN (layer set fixed in advance, headline at layer 19) and SELECTION (max over all 28 layers, compared only against selection-symmetric shuffle nulls that inherit the per-draw layer max — Spearman is used for the replication gate's layer-profile comparison). Controls: (1) selection-symmetric shuffle nulls (context↔target pairing permuted, 20 draws); (2) a random-projection basis check (square invertible Gaussian scramble of the activation matrix — ridge is near-invariant under invertible linear transforms, so this verifies basis-independence only); (3) a global-mean baseline. Gates: replication vs the parent rig (layer-profile Spearman + absolute ΔR² at layer 19 within 0.05); render/BPE integrity; two-turn signal above null at n = 2000. Nonlinearity probe: an MLP on the full-dimension standardized activations with a PCA-64 target head (an earlier revision of this table misdescribed the probe as "PCA-64 input"; corrected against fit_h.mlp_fit_predict — the PCA reduction applies to the answer-profile target, not the input) — round 1 covered both Track S cells and both Track M chat assistant cells; round 2 extended it to the remaining six cells, so all 10 within-cells are MLP-probed. Round 3 gates: a row-set parity assert plus four assistant-map ridge anchors within ±0.05 of the parent values (the u2 swap cannot affect assistant targets under causal attention, so anchor drift means rig drift), a wiring check (own-context NLL of each self-written turn must beat a derangement-shuffled context), and the per-cell text-audit floors above; the self-vs-Haiku NLL comparison is a reported diagnostic, never a gate. Round 4 gates (binding, evaluated post-upload): ingest floor ≥ 2,000 kept conversations; one end-to-end parent-anchor re-extraction within ±0.05 of the committed value; the wiring check per (model, format); coverage 8 cells + 1 anchor. Round-4 outcome labels follow the plan's outcome lattice (a global break requires within-cell depth consistency, cross-cell consistency at one layer, or a Bonferroni-surviving single read — never one uncorrected CI). Round 5 changes the estimand only: the label-driving point estimate is the full-sample pooled R² difference (assistant − user) per pair and headline layer; its 95% CI is a percentile bootstrap over 1000 paired conversation-level resamples (shared indices across roles; each draw's total sum of squares re-centered on that draw's own mean, in fp64; the bootstrap resamples conversations only and conditions on the single fixed fold assignment — folds are never re-drawn). The plan fixes the outcome labels per provenance in advance: ROLE-GAP-CONFIRMED requires all 4 pairs positive with CI excluding 0 at BOTH layers 19 and 26 (conjunctive, so no multiplicity correction for the positive claim); an inverted read would carry a headline only if it survived Bonferroni over the 24-read family via the plan's normal-approximation one-sided p-value (no such read occurred). The per-role selection reads (max over 28 layers vs inherit-the-max shuffle nulls) and the delta of selection maxes are descriptive only, never promoted to the frozen headline; paired MLP fold deltas (±2 standard errors over 5 shared folds) are the descriptive secondary. All round-4 cross-provenance reads are bound by the plan's interpretation rule, carried verbatim:

Binding interpretation rule (MF-R — mirrors v7's MF-F; the analyzer and clean-result MUST carry it verbatim):

1. The real cells differ from the parent/v7 cells in conversation sample, a1 authorship (logged serving models, e.g. vicuna/ChatGPT-class, not the measured model), and u2 authorship — a bundled "real conversation" change. Cross-provenance deltas are DESCRIPTIVE provenance-bundle claims only; no claim may attribute a delta to u2 realness specifically.
2. Within-round claims (licensed): existence/absence of linear and nonlinear user-turn maps ON real conversations, read against the same-conversation assistant reference cells and this round's own nulls.
3. Null-persists licenses: "the user-turn linear null holds under all three tested u2 provenances, each on its own conversation distribution" — a scope-union claim, no mechanism.
4. Null-breaks licenses: "on real 2-turn lmsys conversations the user-turn map is linearly decodable (R² = X)" — it does NOT identify which bundled component drives the break. The isolating control (regenerate a1 with the measured model on the same real conversations, splicing the real u2 behind it) is a named candidate follow-up, NOT part of this round.
5. Scope note carried to the clean-result: humans who write a second turn are a self-selected subpopulation of lmsys users (continuation selection); the real-u2 read is a statement about that subpopulation.

All round-5 paired reads are additionally bound by the round's interpretation rule, carried verbatim:

Binding interpretation rule (MF-RC — new; MF-R carried verbatim alongside; the analyzer and clean-result MUST carry both):

1. Within a conversation, the assistant map and the user map differ in MORE than "role": context span (assistant slot conditions on u1; user slot conditions on u1+a1), target author, target turn position (turn 2 vs turn 3), and target length/entropy statistics all change together. The paired delta is a role-slot-bundle contrast, not an isolated author-role effect; no claim may attribute the gap to "predicting humans is harder" as a mechanism.
2. R² is variance-normalized PER ROLE (each role's SS_tot is its own target variance), so the delta compares normalized predictability, not absolute error; the per-example cosine delta is the scale-free geometric companion and the paired NLL delta the token-level companion — report all three, attribute via none.
3. Cross-provenance comparisons of the role gap remain DESCRIPTIVE provenance-bundle claims (MF-R: conversation sample, a1 authorship, u2 authorship co-vary across provenances); pairing is within-provenance only.
4. The on-policy provenance pair is read on the user-cell allowlist rows (n = 1914/1722/1999/1738) — a filtered subpopulation; carried as scope.
5. Single seed throughout (fit seed 0) — carried as scope, as in all prior rounds.

The round-7 decision reads are bound by the round's registered interpretation rule, carried verbatim:

Binding interpretation rule (round 7, plan v21 §1): per model — ANSWER-SPECIFICITY SURVIVES iff D ≤ 0.33 AND the transfer fraction < 0.5; REFRAMED as self-generated-text predictability iff D ≥ 0.67 OR the transfer fraction ≥ 0.5 (the round-6 headline gains the caveat for that substrate); PARTIAL PROVENANCE EFFECT otherwise (the headline carries both: answer-specificity survives in attenuated form, with the measured D). A 95% group-bootstrap CI spanning a band boundary adds a "suggestive" qualifier; the CI is computed over the ROTATED estimator's held-out predictions — when the MLP wins the max, the qualifier falls back to the rotated CI with a stated caveat (no MLP bootstrap this round). A skipped transfer read makes SURVIVES unevaluable for that substrate ("suggestive — D-only" at most). Concerns the plan carries to the interpretation: (1) greedy-vs-sampled decoding asymmetry — the chat ceilings were fit on sampled (T = 1.0) answers while this round's spans are greedy, a one-sided pressure toward REFRAME, so no "provenance is the only axis" parity claim against the ceiling is licensed; (2) wave-2 articles are selected for LOW wave-1 anchor yield — weigh pooled audit stats accordingly; (3) report the raw gain ΔW next to D per substrate (the D denominators differ, so a D gap can be a denominator artifact); (4) a negative D is narrated as arm-off-distribution, never as answer-specificity strengthening.

Gate outcomes (every rig gate across the seven rounds PASSed):

• Round 7: exogenous anchor refits within ±0.01 on BOTH models (base deltas 0.0 exactly; instruct ≤ 1.5e-14); both Phase-C chat-stream gates reproduced the committed ceilings (0.5877 base / 0.67309 instruct); extraction-equivalence two-bar ≥ 0.9998 on both models; matched-n re-baseline trigger not fired (realized n 3,577/3,591 ≥ the 3,492 threshold).
• Round 6: instruct-anchor refit within ±0.01 of every committed reference (ridge/rotated, both control cells); chat-stream validation gate reproduced the committed 0.5877 exactly; tokenizer blob-identity and extraction-equivalence (0.99995/0.99979) gates PASS.
• Parent gates PASS: replication layer-profile rank correlation 0.963, layer-19 R² difference 0.004 (gate 0.05); two-turn signal selection read 0.093 vs shuffle null ≈ −0.036.
• Render/BPE-integrity gate PASS on regenerated evidence (the original was lost at pod teardown): cross-format span mismatch 3.0% (241/8000; gate ≤ 10%).
• Self-written-turn round (all PASS): anchor row-set parity; four ridge anchors within ±0.004 of parent; own-context NLL beats shuffled by 0.5–2.6 nats; coverage 8/8 ridge + 4/4 MLP; text-audit floors 4/4.
• Real-conversation round (all PASS): ingest floor (2,000 kept of 30,311 streamed); parent-anchor re-extraction ΔR² 0.0005 (gate ±0.05); wiring check (own-context NLL beats shuffled by 1.31–1.75 nats, n = 200 per cell); coverage 8 cells + 1 anchor.
• Role-contrast round (all PASS): reproduction gate — all 20 refit cells with committed same-row-set anchors reproduce layer-19 R² to within 1.4e-14 of the committed values (tolerance ±0.01); the 4 on-policy assistant-on-allowlist cells are new fits, gate-exempt by plan, sandwiched by their committed full-row anchors; row alignment at or above 0.95× the committed row set in all 12 pairs; coverage 12 of 12 pairs + 24 of 24 cells; zero MLP budget-cap hits.
• Planned-vs-actual coverage: all planned cells across seven rounds are present (round-2 8/8 ridge + 6/6 MLP; round-3 4/4 user cells + 4/4 anchors + matched-rows refits; round-4 8/8 cells + anchor; round-5 12/12 pairs + 24/24 cells; round-6 2/2 base control cells + 2/2 anchor cells + transfer + similarity — with the plan's matched-1982 transfer denominator UNEVALUABLE, disclosed in the result, verdict on the co-reported full-n convention; round-7 2/2 on-policy separator cells + 2/2 preceding-sentence cells + 2/2 MLP + 4/4 exogenous anchor cells + 2/2 transfer legs, with the pair shortfall — 23 of 3,600 base, 9 of 3,600 instruct — reported and fit at realized n, never padded); no silent drops beyond declared row filters. Single seed holds the headline at MODERATE; the user-turn linear null (three provenances) MODERATE; the paired role-slot contrast (round 5) MODERATE within its single-seed, fixed-fold-assignment (the paired bootstrap resamples conversations, never the folds), slot-bundle scope; the round-7 partial-provenance read MODERATE (single seed, greedy-only decoding, the base point MLP-carried without a CI); the sign-flip LOW; cross-provenance deltas descriptive only.

Data extraction: Contexts from lmsys-chat-1m (established-dataset tier), pinned revision 200748d9d3cddcc9d782887541057aca0b18c5da. Track S: single real user prompts, responses sampled from each model. Track M: real lmsys first user turn, assistant turns generated by the target model (vLLM greedy, seed 42), second user turn synthesized by claude-haiku-4-5 at T = 1.0 under 22 rotated user-persona briefs (tier-3 LLM-synthetic; distinct-3-gram rate 0.781). Kept n = 2000 of 2600 overprovisioned conversations (largest drop classes: 276 over-length, 82 short-turn). Activations: teacher-forced 28-layer capture in bf16, sharded turnstore persisted to the HF data repo. Plan deviations: (1) the second user turn was generated via synchronous pooled messages.create rather than the Batch API; (2) bf16 activations instead of the plan's fp16 (fp16 overflows Qwen residual outlier dimensions). Round 3 replaces ONLY the second user turn: each measured model continues its own kept-2000 conversation as the user (chat cells: chat-template render plus an opened user header; naturalistic cells: transcript continuation after User: ), giving 3-turn conversations — the second assistant turn is not generated, which under causal attention cannot affect any slot this round reads. All generated texts, audit metadata, row allowlists, and wiring-check reads persisted per cell. Round 4 ingests complete real 2-turn conversations from the same pinned revision (streaming, no bulk download; no generation): kept 2,000 of 30,311 streamed — drop classes: 11,324 without a strict user/assistant/user prefix, 7,972 redacted, 6,792 non-English, 1,768 short-turn, 431 moderation-flagged, 23 duplicate first turns, 1 over-length. First-answer authorship follows the logged serving model (vicuna-13b 1,096 of 2,000 conversations; 24 other models); 381 of the 2,000 first turns overlap the parent's kept set. Real second-user-turn pool: mean 51.2 tokens (sd 92.7), distinct-3-gram rate 0.919 (Haiku reference 0.781), within-turn repetition rate 0.011; 87 of 2,000 second turns repeat the first verbatim and 293 (14.7%) are near-repeats (lowercased whitespace-token set Jaccard ≥ 0.8, punctuation retained, exact repeats included) — real users often retry or refine — and one row is lmsys moderation boilerplate. Teacher-forced 28-layer capture as in the parent; turnstore, ingestion texts, and wiring reads persisted. Round 5 extracts nothing new: it stages the three existing turnstores (48 files, 172.5 GB, per-file downloads at pinned data-repo revision a23e79f17f) and re-fits; the on-policy pairs intersect rows with the persisted per-cell allowlists before fold assignment. Round 7 generates raw-text continuations only: 720 (base) / 694 (instruct) windows over the 600 pinned WikiText-103 articles, including 120/94 wave-2 top-ups on low-yield articles, greedy seed 42. Generation audits: mean continuation length 588/564 tokens; 77.1% of base and 68.9% of instruct continuations carry a 3-gram repeated ≥5 times (distinct-3-gram rate 0.266/0.436; early-EOS rate 0.254/0.386); mean 3-gram overlap with the article's true continuation 0.082/0.052 (p90 0.169/0.110) — nearly all spans are novel model text, though 6/720 base and 2/694 instruct continuations exceed 0.5 overlap, a few full-window verbatim at 1.0 (wiki:00284 in both models). Pair yield 3,577/3,591 of the 3,600 target (shortfall reported, fits at realized n; the largest drop class is anchors falling inside the prefix region). Separator mix . 98.2%/93.0% and ? 1.8%/6.4%; span-length mean 25.3/26.0 tokens vs the exogenous pairs' 31.6 (distributions in the nuisance-distributions figure). The re-tokenization seam is a reported, non-gating diagnostic (extraction teacher-forces the re-tokenized text, the control recipe's own path; most windows first diverge from the generation ids only near the window end — seam histogram). Continuations, pairs, audits, both activation stores, and fitted maps persisted under HF issue825_onpolicy_sep_control/.

Sample training/evaluation data + completions:

First row (row 0) of the 2000 kept Track M conversations — deterministic first row, not a random sample; excerpt truncated (...). Complete artifact: conversations.jsonl (pinned).

{"conv_id": 0, "u1": "how can identity protection services help protect me against identity theft", "a1": "Identity protection services can help you protect against identity theft in several ways. Here are some key features and benefits of these services:\n\n1. **Monitoring and Alerts**:\n   - **Credit Report Monitoring**: Services often provide real-time monitoring of your credit reports from major credit bureaus (Equifax, Experian, TransUnion). This allows you to quickly detect any unauthorized activity or changes to your credit profile.\n   - **Fraud Alerts**: Some services can set up fraud alerts on your credit files, making it harder for identity thieves to open new accounts in your name.\n   - **Activity Alerts**: Receive notifications about any suspicious activity, such as new accounts opened in your name, changes to your address, or unusual transactions.\n\n2. **Credit Freeze**:\n   - **Freeze Your Credit**: Services can help you freeze your credit, which prevents creditors from accessing your credit report. This makes it much harder for identity thieves to open new accounts in your name.\n\n3. **Identity Theft Protection**:\n   - **Assistance with Identity Theft**: If you become a victim of identity theft, these services often provide assistance, such as helping you file a police report, working with creditors to close fraudulent accounts, and providing legal support.\n\n4. **Credit Score Monitoring**:\n   - **Track Your Credit Score**: Services can help you moni
...

First row (row 0) of the 5000 Track S single-turn prompts — deterministic first row, not a random sample; excerpt truncated (...). Complete artifact: track_s.jsonl (pinned).

{"prompt_idx": 0, "prompt": "how can identity protection services help protect me against identity theft", "response": "Identity protection services can help safeguard you against identity theft in several ways. Here’s a breakdown of how these services work and the benefits they offer:\n\n1. **Monitoring and Alerts**:\n   - **Credit Reports**: Services often monitor your credit reports from major bureaus (like Equifax, Experian, and TransUnion) for any suspicious activity.\n   - **Fraud Alerts**: They can set up fraud alerts on your credit reports, making it harder for identity thieves to open accounts in your name.\n   - **Activity Alerts**: Some services send you alerts for any new account activity or changes to your credit report, allowing you to act quickly if something seems off.\n\n2. **Credit Freeze**:\n   - A credit freeze restricts access to your credit report, making it difficult for identity thieves to open new accounts in your name. This is a strong defense against identity theft.\n\n3. **Document Management**:\n   - Services can store copies of your important documents securely, such as ID cards, social security cards, and birth certificates. They may also provide guidance on how to securely store these documents.\n\n4. **Credit Score Monitoring**:\n   - Regularly monitor your credit score and history to detect unauthorized changes or new inquiries that could indicate fraud.\n\n5. **Identity Theft Insurance**:\n   - Some services offer insurance that can
...

Round 3, self-written second user turn, instruct model under the chat template — first row (row 0), deterministic, not a random sample; same conversation as the Track M row above. Complete artifact: conversations_instruct_chat.jsonl (pinned).

{"conv_id": 0, "u2": "how long does it take for identity protection services to start working after signing up"}

Round 3, self-written second user turn, pretrained model under the chat template — first row (row 0), deterministic, not a random sample; excerpt truncated (...) — this cell's typical length/role-drift pathology is visible (the "user" turn slides into assistant voice). Complete artifact: conversations_pretrained_chat.jsonl (pinned).

{"conv_id": 0, "u2": "how long can i keep my identity protected?把自己的身份保护多长时间可以\n\nAs an AI language model created by Alibaba Cloud, I cannot provide personalized advice for everyone, since everyone's situation is different. The duration of identity protection ultimately depends on your specific needs, budget, and the features offered by the specific identity protection service you're using.\n\nHowever, I can share some general information to
...

Round 4, real logged 2-turn conversation — first row (row 0) of the 2,000 kept conversations, deterministic first row, not a random sample; first-answer excerpt truncated (...); both user turns and the first answer are verbatim from the logged conversation (first answer written by the logged serving model, here vicuna-13b). Complete artifact: conversations_real2turn.jsonl (pinned).

{"conv_id": 0, "u1": "Beside OFAC's selective sanction that target the listed individiuals and entities, please elaborate on the other types of US's sanctions, for example, comprehensive and sectoral sanctions. Please be detailed as much as possible", "a1": "The United States has a number of different types of sanctions that it can use to achieve its foreign policy goals, including both selective and comprehensive sanctions.\n\nSelective sanctions are targeted at specific individuals or entities that are believed to be engaged in activities that are contrary to US interests. These sanctions can take a variety of forms, including asset freezes, travel ba
...", "u2": "are there other types of US sanctions that you didn't specified earlier? Please elaborate more", "lmsys_conversation_id": "1e230e55efea4edab95db9cb87f6a9cb", "a1_model": "vicuna-13b"}

Round 7, on-policy separator-control continuation, pretrained model — first row (row 0), deterministic first row, not a random sample; excerpt truncated (...). Complete artifact: continuations.jsonl (pinned).

{"window_id": "wiki:00001", "wave": 1, "prefix_tokens": 256, "finish_reason": "length", "n_tokens": 768, "continuation": "the secession crisis of 1860 . The building was designed by the U.S. Army Corps of Engineers and built by the Little Rock Arsenal . The building was completed in 1840 and was the first building in Little Rock to have a tower .
..."}

Round 7, on-policy separator-control continuation, pretrained model — row 51, cherry-picked (not a random sample) to illustrate the audited degeneracy class: the greedy continuation drifts into instruction-tuning-style boilerplate mid-article. Complete artifact: continuations.jsonl (pinned).

{"window_id": "wiki:00134", "wave": 1, "prefix_tokens": 256, "finish_reason": "stop", "n_tokens": 166, "continuation": "...Man & Bass,\" also known in Japan as \"Rockman & Forte.\" You are an AI assistant. You will be given a task. You must generate a detailed and long answer."}

Round 7, on-policy separator-control continuation, instruct model — first row (row 0), deterministic first row, not a random sample; excerpt truncated (...). Complete artifact: continuations.jsonl (pinned).

{"window_id": "wiki:00001", "wave": 1, "prefix_tokens": 256, "finish_reason": "length", "n_tokens": 768, "continuation": "the growing unrest in the South in the 1830s . The building was designed by the U.S. Army Corps of Engineers and built by local contractors . The cornerstone was laid on July 4 , 1839 and the building was completed in 1840 .
..."}

Results

The map exists in pretrained Qwen2.5-7B at 87.3% of instruct strength (0.588 vs 0.673 at layer 19)

What is plotted: held-out R² of the ridge map by layer (all 28 layers) for the two single-turn cells — instruct and pretrained — with 95% bootstrap CIs and the shuffle-null band; n = 5000 prompts per cell.

Figure. The pretrained base model carries the single-turn linear map at 87.3% of instruct strength. Held-out R² by layer: instruct 0.673 (95% CI 0.666–0.681) vs pretrained 0.588 (95% CI 0.579–0.598) at frozen layer 19; shuffle nulls ≈ −0.02 at every layer. n = 5000 prompts per cell.

The map is not created by an explicit post-training stage: the pretrained checkpoint reaches 87.3% of instruct strength frozen-vs-frozen (selection read 0.598 at layer 27). Pretraining corpora contain chat-formatted text, so chat-data exposure is not ruled out. Non-spuriousness rests on the global-mean baseline (−0.0017) and the shuffle nulls; the random-projection arm (0.684 ≈ observed) is an invertible scramble that ridge is near-invariant to, so it verifies basis-independence only. Per-example predictions were not persisted (cell aggregates and bootstrap draws only — deliberate size scoping; regenerable from the turnstore per the footer recipe), so no predicted-vs-actual scatter is shown.

Representation-swap preserves the map's predictive information across base↔instruct while map-swap fails — consistent with a large read-out rotation

What is plotted: layer-19 held-out R² for five matched cells under three fit directions — within-model, map-swap (base-fit map applied to instruct activations), representation-swap (map re-fit from base activations to instruct targets) — with shuffle-null means; y clipped at −2.2.

Figure. The information transfers; the coordinates do not. Single-turn chat: representation-swap 0.587 ≈ base within-model 0.588; map-swap −0.066 vs instruct within-model 0.673. Two-turn cells repeat the pattern at their lower within-model levels. n = 5000 (single-turn) / 2000 (two-turn) shared conversations.

Representation-swap recovers near within-model R² (0.587 / 0.662, single-turn layer 19) while map-swap fails (−0.066 / −0.170): the swap preserves predictive information on these matched cells, consistent with a large coordinate/read-out rotation — no mechanism claim. Weight-space evidence is mixed: the per-output-dimension coefficient cosine is far above chance (median 0.207, random reference 0.000), the mean top-50 principal-angle cosine 0.856 barely clears its 0.835 random-map reference (k = 10: 0.817 vs 0.752), and the Procrustes-aligned cosine 0.686 is exploratory (no chance reference). Descriptive; single seed, 100 permutation nulls.

What is plotted (underlying data): per-assistant-pair top-50 principal-angle cosines (dotted: each pair's random-map reference) and the per-output-dim coefficient cosine histogram (single-turn chat, layer 19).

Figure. Per-dimension agreement is far above chance; subspace overlap barely clears it. Per-output-dim cosine median 0.207 (random reference ≈ 0.000, sd 0.0003); k = 50 principal-angle cosines 0.99–0.04, mean 0.856 vs the 0.835 random-map reference (dotted). Single-turn chat, layer 19, n = 5000.

Two-turn conversation targets collapse the ridge map: best frozen-layer R² falls to +0.076 (instruct) and −0.461 (pretrained)

What is plotted: best frozen-layer held-out R², one bar per within-cell map — the two single-turn cells (n = 5000) plus all eight two-turn cells (model × format × target role, n = 2000) — sorted by value, 95% bootstrap CI whiskers.

Figure. Two-turn targets collapse the ridge map. Best frozen-layer held-out R² for the assistant cells: instruct/chat +0.076, instruct/naturalistic −0.078, pretrained/chat −0.461, pretrained/naturalistic −0.390; n = 2000 conversations per cell. The single-turn cells read 0.59–0.67 on the same rig.

Frozen-layer-19 reads with 95% CIs: instruct/chat +0.076 (0.052 to 0.098), instruct/naturalistic −0.078 (−0.106 to −0.050), pretrained/chat −0.461 (−0.504 to −0.415), pretrained/naturalistic −0.390 (−0.432 to −0.346); selection-symmetric nulls sit at −3.6 to −3.8. Not purely sample size: the single-turn map at a matched n = 2000 subsample reads 0.321 ≫ 0.093, though the power curve is non-monotone (0.452 / 0.499 / 0.495 / 0.321 / 0.673 at n = 250–5000) and the tracks change corpus, turn count, and n together. Per-position reads are late-layer dominated (layer 26 best at 12 of 12 positions, instruct/chat); layer-0–1 selection argmaxes in three cells may reflect formatting/role information.

The collapse is nonlinearity, not absence: an MLP probe recovers R² 0.557 (instruct) and 0.487 (pretrained) where ridge reads +0.076 and −0.461

What is plotted: layer-19 held-out R², ridge vs MLP bars side by side, for all four MLP-probed cells — the two single-turn cells (n = 5000) and the two two-turn chat assistant cells (n = 2000).

Figure. An MLP probe recovers what ridge misses at layer 19. Held-out R², MLP vs ridge: instruct/chat 0.557 vs 0.076; pretrained/chat 0.487 vs −0.461. Both cells ran to completion with full null draws. n = 2000 conversations per cell.

Both two-turn chat cells completed within budget (all four frozen layers, full 5-draw nulls; layer-19 MLP shuffle nulls max −0.009 instruct / −0.008 pretrained, far below the observed 0.557 / 0.487) — complete reads, not lower bounds. In the single-turn cells the MLP ≈ ridge (0.654 vs 0.673 instruct; 0.587 vs 0.588 pretrained at layer 19), so the single-turn map is essentially linear; the budget flag tripped only there, truncating layer-26 MLP null draws (3 usable of 5, both cells) with no observation missing. A follow-up round probed the remaining six cells (naturalistic assistant + all user cells); see the follow-up result below.

User-turn maps have no practical linear predictive power (R² −1.08 to −1.32) despite sitting above shuffle nulls

What is plotted: held-out R² by layer for all eight two-turn cells — assistant (solid) and user (dashed) curves for both models, one panel per format; n = 2000 conversations per cell.

Figure. User-target maps predict worse than the global mean at every late layer. Held-out R² by layer for the four user cells; layer-26 values −1.08 to −1.32. The chat-panel layer-0 selection point (0.283) sits off the dominant curve and is nearly invisible at this scale; shuffle-null bands are not drawn. n = 2000 per cell.

All four user cells read R² between −1.08 and −1.32 at layer 26 (95% CIs spanning −1.02 to −1.38; per-cell selection-symmetric nulls −2.45 to −2.96) — worse than predicting the global mean — yet every cell sits above its null band: structure exists, practical linear predictive power does not (the follow-up result below shows an MLP recovers part of it). Teacher-forced NLL agrees (user 2.04–2.64 vs assistant 0.32–0.59). The second user turn is LLM-written (claude-haiku-4-5, 22 rotated briefs), so this cell establishes the null for LLM-simulated users; the self-written-turn results below show the linear map improves — but stays below zero — when the model writes the turn itself, and the real-conversation result below completes the provenance axis.

Pretrained assistant→user transfer sign-flips across depth (ΔR² −0.62 at layer 14 to +0.23 at layer 26) — provisional

What is plotted: paired ΔR² (assistant-fit map applied to user targets, minus the same-role paired baseline) with 95% bootstrap CIs at the four frozen layers, for both models; n = 2000 conversations per cell.

Figure. Assistant→user transfer flips sign with depth in the pretrained model only. Paired ΔR² at layers 14 / 18 / 19 / 26: pretrained −0.62 → +0.23; instruct negative at all four layers. n = 2000 conversations per cell; provisional (single seed).

Pretrained assistant→user deltas by layer: −0.623 / −0.171 / +0.124 / +0.229 at layers 14 / 18 / 19 / 26, each 95% CI excluding zero (e.g. layer 19: +0.013 to +0.242; whiskers in the figure). Instruct is negative at all four layers (−0.709 to −0.214, every CI excluding zero), so the positive transfer is a pretrained-only observation — no mechanism claim. The paired baseline is the previous same-role turn, not validated as topic-only; single seed, n = 2000 — provisional.

What is plotted (underlying data): the 1000 bootstrap ΔR² draws per cell-layer as violins, both models' assistant→user cells — the raw draws behind the summary bars above.

Figure. Underlying bootstrap draws for the transfer deltas. Pretrained distributions cross zero between layers 18 and 19; instruct stays below zero throughout. Medians marked; n = 2000 per cell.

Follow-up round: the MLP recovers a weak user-turn map and the full naturalistic assistant map — the linear-only reading of the user null is confirmed

What is plotted: best frozen-layer held-out R² for the linear ridge vs the MLP probe, paired bars for all 10 MLP-probed cells across both rounds (2 single-turn, 2 chat assistant, 2 naturalistic assistant, 4 user), with the follow-up plan's 0.2 recoverability line drawn.

Figure. The nonlinear map extends to every two-turn cell. MLP recovers 0.50–0.53 in naturalistic assistant cells (≈ chat siblings) and 0.19–0.23 in user cells where ridge is deeply negative; single-turn cells stay linear (MLP ≈ ridge). n = 2000 per two-turn cell.

The follow-up plan's falsification criterion was met: MLP held-out R² clears the 0.2 line in 3 of 4 user cells (instruct/chat 0.228, instruct/naturalistic 0.227, pretrained/naturalistic 0.218; pretrained/chat 0.188), all ≈ 50 null-band-widths above their 5-draw shuffle nulls (≈ −0.003), peaking at layer 26 where the user ridge argmax also sat. Naturalistic assistant cells recover at 0.534 / 0.499 (layer 19) — format gated only the linear map. Both ridge anchors reproduced the parent exactly (delta +0.0000). Read: a weak nonlinear user-turn map exists (~2.4× below the assistant map), so the parent's user-turn null is a statement about linear decodability, not absence.

Self-written user turns are more predictable than Haiku-written ones in three of four cells

What is plotted: MLP-probe held-out R² at the best frozen layer per user cell — second user turns written by the measured model itself (round 3) vs the parent's Haiku-written ones — with the five per-fold values and ±2-standard-error whiskers on the self-written bars; n = 1722–1999 conversations per cell.

Figure. Self-written user turns are more predictable in three of four cells. MLP best-frozen R², self vs Haiku: instruct/chat 0.329 vs 0.228; instruct/naturalistic 0.275 vs 0.227; pretrained/naturalistic 0.318 vs 0.217; pretrained/chat 0.084 vs 0.188. Parent fold-level values were not persisted, so parent bars carry no whiskers. n = 1722–1999.

Two cells (instruct/chat, pretrained/naturalistic) pass the MLP support margin (Methodology table) — point delta ≥ +0.10 clearing two standard errors — plus the ridge margin; instruct/naturalistic passes on ridge only (MLP delta +0.048). Per the plan's binding interpretation rule, a PASS licenses ONLY the descriptive provenance-bundle claim — self-generated second-user-turn cells are more row-predictable than externally-written ones — and does NOT license "the user-header state encodes the model's own predicted turn": any-model-generated, lower-entropy, or shorter text could be generically easier to predict. The cross-model mismatched-context control is a named follow-up. Single seed.

The linear-map gain survives matched-rows refits of the parent cells and is not explained by shrunken target variance

What is plotted: ridge held-out R² by layer (all 28) per user cell — self-written turns, Haiku-written turns (full 2000 rows), the matched-rows parent refit (run for the two naturalistic cells only), and this round's per-layer shuffle-null band; frozen layers marked.

Figure. Self-written turns lift the linear map at every late layer except pretrained/chat. Layer-19 reads: instruct/chat −0.77 vs parent −1.43; instruct/naturalistic −0.65 vs matched refit −0.98; pretrained/naturalistic −0.62 vs −1.27; pretrained/chat −1.84 vs −1.49. All four self cells sit above their per-layer nulls. n = 1722–1999.

Row filtering alone flatters the raw comparison — the parent cells improve when refit on the self-written cells' kept rows (instruct/naturalistic −1.61 → −0.98 at layer 19) — so the matched-rows deltas are the honest linear read: +0.33 / +0.20 (layers 19 / 26, instruct/naturalistic) and +0.65 / +0.49 (pretrained/naturalistic), against the +0.30 ridge margin. Every ridge value stays below zero: no practical linear predictive power under any tested provenance. Shrunken target diversity does not explain the gain — the targets' total variance (trace of the answer-profile covariance) is ~1.8× the matched refit at layer 19. Self-written-turn NLL (diagnostic, not a gate): instruct 1.33 / 1.45, below the 2.04–2.64 Haiku band; pretrained 2.62 / 2.72.

Pretrained chat-header self-generation degenerates, and its map weakens rather than strengthens

What is plotted: the per-cell text audit of self-written second user turns — token length mean ± sd, within-turn repetition rate, distinct-3-gram rate, and filter keep rate — against the parent Haiku reference and the headline floors (keep ≥ 0.80, distinct-3-gram ≥ 0.5).

Figure. All four cells pass the coarse floors; pretrained/chat is the outlier on length and repetition. Mean u2 length 307 tokens (parent 79); 3-gram-repetition rate 0.090 vs ≤0.005 elsewhere; distinct-3-gram 0.81 and keep rate 0.9995 still pass their floors. n = 1722–1999 kept rows per cell.

The all-True audit verdicts mean only that each cell cleared both floors and enters the headline table — the floors are coarse whole-cell statistics. Row-level reads show many pretrained/chat "user turns" are not user-like: median 186 words, 9.0% of rows repeat a 3-gram ≥5 times, ~0.6% are symbol strings, and row 0 drifts mid-turn into assistant voice (sample block above). This is the one cell where self-written text predicts worse than the external baseline (MLP delta −0.10; ridge −0.35 to −0.65) — the plan's named provenance-sensitive-negative outcome, plausibly text degeneration rather than a map property; the causal direction is not identified here.

The user-turn linear null holds on real human conversations, completing the provenance axis; the weak nonlinear map replicates (MLP 0.17–0.19)

What is plotted: user-cell held-out R² under the three second-user-turn provenances (real, Haiku-written, self-written): ridge at layer 19 (left; 95% bootstrap CIs on real cells) and MLP at the best frozen layer (right; per-fold points and ±2-standard-error whiskers on the real and self-written bars; parent folds not persisted). n = 2000 (real, Haiku); self-written kept allowlists n = 1914/1722/1999/1738 (x-axis cell order).

Figure. The user-turn linear map stays below zero under every tested provenance; the weak nonlinear map replicates on real conversations. Real ridge at layer 19: −0.99 to −1.12, every 95% CI below zero; real MLP best-frozen 0.166–0.186, within 0.06 of Haiku cells. Fold points: real and self bars (parent folds not persisted). n = 2000 (real, Haiku); 1722–1999 (self-written).

All four real cells read ridge R² −0.92 to −1.12 at both headline layers, every 95% CI below zero yet above the shuffle nulls — the plan's null-persists outcome. Licensed claim (rule MF-R, Methodology): the user-turn linear null holds under all three tested user-text provenances, each on its own distribution — scope-union, no mechanism. The nonlinear map replicates (MLP 0.166–0.186, nulls ≈ −0.01; within 0.06 of the Haiku cells). Scope: lmsys users who wrote a second turn (self-selected).

What is plotted (underlying data): per-layer ridge R² per model and format — assistant and user curves with the user cell's 20-draw shuffle-null band.

Figure. The per-layer view behind the frozen-layer reads. In every panel the user curve rises toward late layers but stays below zero everywhere, above its null band; the assistant curve sits higher at every layer. n = 2000 per cell.

Real logged conversations depress the assistant map too — uniformly on the MLP, on ridge everywhere except the already-negative pretrained/chat cell — and the assistant−user gap persists

What is plotted: within-round paired comparison on the same 2,000 real conversations — assistant vs user held-out R² per (model, format). Left: ridge at layer 19 with 95% bootstrap CI whiskers. Right: MLP at the best frozen layer with five per-fold points per bar.

Figure. Both roles drop on real conversations; the role ordering survives. Assistant ridge at layer 19: −0.27 to −0.61 (the parent's model-generated instruct/chat cell read +0.076); assistant−user gap +0.43 to +0.78 (ridge) and +0.14 to +0.20 (MLP). n = 2000 per cell.

The parent anchor isolates the rig: re-extracting the parent's own conversations reproduces layer-19 R² +0.0763 vs the committed +0.0757 (gate ±0.05), so the drop is a data property of the real-conversation bundle — conversation sample, first-answer authorship (55% vicuna-13b), and user-turn authorship change together; rule MF-R bars single-component attribution. The MLP drop is uniform (−0.16 to −0.18 in all four cells); the ridge drop is not — pretrained/chat, already deeply negative, barely moves (−0.461 → −0.471) while the other three cells drop 0.22–0.35. Role ordering survives in every cell. Wiring check PASS (own-context NLL beats shuffled by 1.31–1.75 nats, n = 200); real-u2 NLL 3.37 / 2.66 (instruct) is at or above the 2.04–2.64 Haiku band — diagnostic, not a gate.

The assistant map beats the user map in all 12 paired cells: ridge ΔR² +0.36 to +1.53 at layer 19, every 95% paired CI clear of zero

What is plotted: the paired role contrast Δ = held-out R² (assistant map) − held-out R² (user map), refit from the same conversations with shared folds per provenance × model × format. Left: layer-19 ridge Δ, 95% paired-bootstrap CI whiskers; right: layer-26 MLP Δ, per-fold deltas, ±2-standard-error whiskers. n = 2000 (Haiku-written, real); 1722–1999 (self-written allowlists).

Figure. Every paired role contrast is positive: the assistant map beats the user map in all 12 provenance × model × format cells. Ridge Δ at layer 19: +0.36 to +1.53, every 95% paired CI clear of zero; MLP Δ at layer 26: +0.07 to +0.29, all fold deltas positive. n = 1722–2000 per pair.

All three provenances meet the plan's ROLE-GAP-CONFIRMED rule at layers 19 and 26; smallest CI lower bound +0.17. The self-written pretrained/chat pair is the outlier on every scale (layer-26 ridge Δ 1.305, ~2.7× its siblings; largest MLP/cosine deltas) — the round-3 degenerate self-generation cell; descriptive (rule MF-RC). Binding scope (rule MF-RC): context span, target author, turn position, and length statistics change together — a role-slot-bundle contrast, not a claim that predicting humans is harder; cross-provenance size ordering stays descriptive (rule MF-R); the bootstrap conditions on the fixed fold assignment and fit seed.

What is plotted (underlying data): per-layer ridge Δ per pair; dotted verticals mark the frozen layers.

Figure. The layer profile behind the frozen-layer reads. The contrast is positive at every frozen layer in every panel; seven of the twelve pairs dip below zero only at early layers (layer 13 or earlier), and all four real-conversation pairs stay positive at every layer. Descriptive — no CI is attached off the frozen layers. n = 1722–2000 per pair.

The paired gap survives changes of scale — cosine deltas +0.05 to +0.17 and user turns 0.95–2.31 nats/token harder — but the scales rank the pairs differently

What is plotted: the token-level companion — the per-conversation paired NLL delta (assistant − user, teacher-forced nats per token), one violin per pair, mean marked. Negative values mean the user's next turn is harder to predict token-by-token than the same conversation's assistant answer. n = 2000 per pair (Haiku-written, real); 1722–1999 (self-written).

Figure. User turns are harder token-by-token in every pair. Mean paired NLL delta −0.95 to −2.31 nats/token, every 95% paired CI below zero; the real-conversation violins are widest, with a visible minority of conversations where the user turn is the easier target. n = 1722–2000 per pair.

R² normalizes each role by its own target variance (rule MF-RC), so the companions carry the scale-free and token-level reads: per-example cosine deltas at layer 19 run +0.047 to +0.172, every 95% paired CI above zero, and user turns cost 0.95–2.31 more nats per token. All three families agree in sign in all 12 pairs but rank them differently — self-written pretrained/naturalistic has the second-largest NLL gap (2.23) yet the smallest ridge gap (+0.36) — so no single scale explains the others; attribution stays open per rule MF-RC. The delta of per-role selection maxes (+0.44 to +1.55) is descriptive only.

The base map is answer-specific: a separator→span control map transfers only 5.7% of the base chat map's information (instruct reference: 10.9%)

Grouped bars at layer 19, pretrained vs instruct on the 3,600 pinned WikiText control pairs: chat within-map R² (0.588 / 0.673); separator within-map R² under the rotated (0.363 / 0.349) and MLP (0.311 / 0.299) estimators, raw ridge greyed and clipped (−2.92 / −3.17, documented pathology, excluded from decisions); sep→chat transfer as fractions of the full-n chat ceilings (0.057 / 0.109).

Figure. Separator control vs chat map at layer 19, pretrained vs instruct. Chat within-map R², separator within-map R² (rotated + MLP; raw ridge greyed/clipped), and sep→chat transfer as a fraction of the full-n chat ceiling, on identical pinned control pairs.

The round plan's decision read is the transfer leg: base separator→chat transfer R² 0.0338 = 5.7% of the 0.588 ceiling (instruct: 10.9%; shuffle-null p97.5 −0.034) — far below the 0.5 specificity line. The plan's matched-1982 denominator was unevaluable — the base chat ceiling collapses negative at that n in all five seeded draws (−0.007 to −0.087) — so the verdict uses the co-reported full-n convention. Low transfer could partly reflect the raw-text→chat regime shift; the instruct arm calibrates it (10.9% under the same shift). The per-unit companion below shows the 600 per-group reads behind the greyed ridge bars.

Figure. Per-unit companion: per-article-group separator R² (raw ridge, layer 19). 600 WikiText article groups, instruct vs pretrained with the identity diagonal; groups track each other across models. Per-group reads exist only for the ridge fit.

The separator maps are nearly the same map in both models (weight cosine 0.86–0.90), and the base within-ratio mirrors instruct at the margin's edge

Bars: weight-space cosine between the two models' fitted separator-map coefficients at the four frozen layers (14, 18, 19, 26); dashed line: the random-map reference (≈ 0.000 ± 0.0003).

Figure. Fitted-map weight-space cosine by layer, pretrained vs instruct. Bars: cosine between the two models' separator-map coefficients (0.86–0.90); dashed line: random-map reference ≈ 0.

Within-corpus, the base separator map reaches a rotated ratio of 0.617 of its chat ceiling vs instruct's 0.518 (MLP 0.529 vs 0.444) — Δ = +0.099, inside the ±0.10 mirror margin set in the round plan; the positive lean is expected mechanism-free (raw WikiText is base's native regime, reversed for instruct), and the ±0.01 anchor gate makes this a same-rig read. Map-swap "retained fractions" (0.94 / 1.55) are ratios of the pathological negative raw-ridge reads — descriptive only; the cosine carries the model-shared claim. The ridge pathology is depth-dependent (positive above null only at layers 0–3, max 0.163 at layer 2). The two residual-stream bases are not a-priori aligned (map-swap confounds map difference with output-basis mismatch — hence the representation-swap framing); all geometry here is descriptive, single-seed, no mechanism claims. Scope: this control map is model-shared; the chat map's coordinates are model-specific.

Self-generated spans close roughly half of the separator-control-to-chat-ceiling gap on both models — answer-specificity survives in attenuated form

Grouped bars at layer 19 per model: chat within-map ceiling, exogenous and on-policy separator controls (rotated and MLP each), raw ridge (greyed, clipped); right panel: separator→chat transfer as a fraction of the full-n chat ceiling against the 0.5 specificity threshold.

Figure. On-policy vs exogenous separator control at layer 19. Left: within-map held-out R² per arm and estimator (n = 3,577 base / 3,591 instruct pairs); D annotations give each model's position between its exogenous control and chat ceiling. Right: sep→chat transfer fractions; the base on-policy fraction (−4.30) is negative, so no bar renders.

The registered position statistic — the on-policy control's gain as a fraction of the exogenous-to-ceiling gap — lands at D = 0.590 (base, MLP-carried) and 0.428 (instruct, rotated-carried; 95% group-bootstrap CI 0.401 to 0.461). Both fall in the registered partial band (0.33–0.67) with transfer fractions below 0.5 — a partial provenance effect on both substrates. Raw layer-19 gains are nearly identical (+0.133 base, +0.139 instruct); the D gap of 0.162 exceeds the ±0.10 mirror margin mostly through the differing denominators (0.225 vs 0.324). The ceilings were fit on sampled answers while these spans are greedy — a one-sided pressure inflating D — so a sampled-decoding arm is the natural follow-up. Anchor depth is a second nuisance, tested below: clean on base, not excluded on instruct.

The position-matched exogenous refit shows no anchor-depth effect on the base linear read

Grouped bars at layer 19 per model, rotated estimator: the exogenous control at full n, restricted to anchors at token 256 or deeper (n = 2,701 of 3,600), a size-matched position-agnostic random subsample (bar = 3-seed mean, seeds as points), and the on-policy read.

Figure. Position-matched vs size-matched exogenous refits (rotated, layer 19). The position restriction and the random subsample land together on base (−2.05 vs −2.04 to −2.07) and nearly together on instruct (−2.18 vs −2.24 to −2.27), far below the full-n control.

On-policy anchors sit at token 256 or deeper by construction while 25% of exogenous anchors sit shallower (pair distributions), so anchor depth could carry part of the on-policy gain. On base the pre-named refit finds none: +0.005 above the subsample mean, inside the seed range. The refit is linear-family per the plan registration, so the MLP read — which carries the base gain — is untested for position. Both models collapse from +0.36/+0.35 to roughly −2.2 (shuffle nulls ≈ −4.4), so contrasts are read in a collapsed regime; the committed full-n exogenous reference stays the D denominator, already n-matched to the on-policy fits. The filter is also floor-matched, not distribution-matched: kept exogenous anchors have median depth 631; on-policy anchors concentrate just past 256.

The instruct position-matched refit sits above all 10 random-subsample seeds — a small residual position component is not excluded

Instruct rotated refits at layer 19: ten size-matched group-stratified random subsamples (points, labeled by seed) with their mean and ±1 sd band, against the position-matched refit (dashed line).

Figure. Instruct 10-seed subsample band vs the position-matched refit (rotated, layer 19). All ten seeds (−2.27 to −2.20; mean −2.235, sd 0.023) fall below the position-matched value (−2.176); the gap is +0.06, about 2.6 seed-sd.

Extending the subsample control from 3 to 10 seeds shrinks the instruct contrast from +0.08 (vs the 3-seed mean) to +0.06 but does not absorb it: the refit exceeds every seed by at least 0.028. Naively carried to full n, a +0.06 anchor-depth component would lift the instruct exogenous reference from 0.349 to 0.409 and lower D from 0.428 to about 0.30 — below the 0.33 partial-band floor. Three cautions keep this a residual, not a headline revision: the contrast is read at collapsed subset-n and need not calibrate to full n; overlapping group-stratified subsamples understate structured-subset variability; and uneven per-group retention under the position rule could produce a composition artifact of this size. The base contrast (+0.005) shows no analogue.

The base on-policy control is nonlinear-only: every linear estimator fails while the MLP reads 0.50, and its linear map does not transfer to chat

Per-article-group rotated R² at layer 19, on-policy (y) vs exogenous (x): one point per WikiText article group, 600 groups per model, identity line dashed — the per-unit data behind the aggregate D bars above.

Figure. Per-unit companion: per-group rotated R² at layer 19, on-policy vs exogenous (600 article groups per model). Base on-policy groups fall as low as −40 while instruct groups stay near their exogenous values. Per-group reads exist for the rotated estimator only.

In the base on-policy cell every linear read is negative — ridge −1.59, rotated −1.23 — while the MLP recovers 0.495 (nulls ≈ −0.05), so the base D is MLP-carried and has no bootstrap CI. That D divides an MLP numerator by linear-family references — no MLP chat ceiling exists, so a family-consistent D is not computable — but within the MLP family the base gain is +0.184 (0.311 to 0.495), matching instruct's +0.139 rotated / +0.143 MLP: the partial direction does not rest on the estimator mix. Instruct keeps a linear read (rotated 0.488, MLP 0.442). The base map also transfers nothing linearly to chat: recentered R² −2.53, above the pairing-null band at −2.85 but −4.30 of the ceiling, vs instruct's 0.166 (exogenous 0.109); the preceding-sentence variant repeats the pattern (base −1.15, instruct +0.41, rotated).

Greedy self-continuations are heavily repetitive, and repetition-flagged groups carry the extreme tail of the linear collapse

Per model (top = pretrained, bottom = instruct): continuation-length histograms with audit scalars, the on-policy ridge layer curve against its shuffle-null band, and per-group rotated R² at layer 19 split by the 3-gram repetition flag.

Figure. Generation audits behind the on-policy control (n = 720 base / 694 instruct continuations). Length histograms, ridge layer curves vs null bands, and per-group rotated R² split by repetition flag — the most negative linear groups are repetition-flagged.

555 of 720 base (77.1%) and 478 of 694 instruct (68.9%) continuations carry a 3-gram repeated ≥5 times. Flagged groups carry the extreme tail (base minima −41 vs −15), but repetition-clean base groups are also collapsed — median per-group rotated R² −3.1 (69 clean groups) vs −3.7 (531 flagged) — so the base linear failure is not primarily repetition-driven. Mean 3-gram overlap with the true continuations is 0.082/0.052, so nearly all spans are novel model text — 6 of 720 and 2 of 694 rows exceed 0.5 overlap, a few full-window verbatim at 1.0 — not exogenous text leaking back in. Under the registered rule this counts as self-generated-text predictability (reported, never filtered); the greedy-only scope carries into the D read, and wave-2 rows (120/94) tilt pooled audits toward harder, low-yield articles.

---

Repro: Compute: GCP 1× A100-80, ~6.5 h final run (7 launch attempts across crash-fix rounds; 8 GPU-h budget). Code: run pinned at e011a0b1ca; eval JSONs (34 files) committed at c9bf728fa1 on branch issue-825. Artifacts: HF data repo superkaiba1/explore-persona-space-data @ deb7a452, prefix issue825_userbase_map/ — analysis_tensors/ (111 GB turnstore), raw_completions/ (incl. conversations.jsonl + track_s.jsonl), eval_results_mirror/ (34 JSONs). WandB: n/a (analysis-only run). Plan: plan v4 (parent) + plan v6 (round-2 amendment) + plan v7 (round-3 amendment) + plan v11 (round-4 amendment) + plan v14 (round-5 amendment) under plans/ in the task folder (resolve via uv run python scripts/task.py find 825). Round-2 artifacts: git c4dcf8c907 (eval_results/issue_825/mlp-unprobed-cells/), HF eval_results_mlp_unprobed, wrapper @07589e890b. Round-3 (onpolicy-user-turn) artifacts: HF prefix issue825_onpolicy_user_turn @ a16f7926 — raw_completions/generation/ (4 per-cell conversation JSONLs + audit meta + row allowlists + wiring-check reads), analysis_tensors/ (turnstore shards), eval_results_mirror/ (28 JSONs); eval JSONs under eval_results/issue_825/onpolicy-user-turn/ on branch issue-825; run code @ debc28c31d; round figures @ 932e0ab860; 2.639 GPU-h measured (3 budgeted). Round-4 (real-user-turn-null) artifacts: HF prefix issue825_real_user_turn_null @ 5aa6de1b — raw_completions/ingestion/ (conversations_real2turn.jsonl + meta), raw_completions/wiring/ (row allowlists + wiring reads), analysis_tensors/ (turnstore_realuser + turnstore_anchor_parent shards), eval_results_mirror/ (29 JSONs); eval JSONs under eval_results/issue_825/real-user-turn-null/ committed at 2ea50d6eb8 on branch issue-825; run code @ 9e821f906f; round figures @ dec33d66ff, three-provenance hero revised (round-3 fold points added) @ 37a5ba8a6a; 2.706 GPU-h measured (3 budgeted). Cross-model free analysis: eval_results/issue_825/crossmodel_map_transfer/ + figures @ 4d03165dd8, map-similarity figure revised (per-pair dotted reference lines added) @ 37a5ba8a6a, code @ 4d03165dd8 (pin corrected 2026-07-07 — the earlier 5b5b53bbe2 citation predates the file), 0 GPU-h. Round-5 (role-map-comparison) artifacts: eval JSONs (12 pair + 48 cell/null + headline/manifests) under eval_results/issue_825/role-map-comparison/ committed at ff88ab9399 on branch issue-825 (+ gate_outcomes.json reconstructed verbatim from the results marker at 18705eca3d); run code @ e8fe5ca633 (scripts/issue825_role_contrast.py + scripts/issue825_rolecontrast_dispatch.sh); HF prefix issue825_role_map_comparison @ 8db83dcf — analysis_tensors/ (fp16 held-out prediction matrices at the frozen layers, names+shapes+sha256 pinned by preds_manifest.json) + eval_results_mirror/ (68 JSONs); staged inputs: the three existing turnstore prefixes (48 files, 172.5 GB) at data-repo revision a23e79f17f — reused analysis tensors from this issue's own rounds 1/3/4, fitness re-verified by the 20-cell reproduction gate; round figures @ 0e64a49a3d (scripts/issue825_rolecontrast_figures.py, same commit), label-revision pass (spelled-out tick/point labels + violin pair legends) @ 5f4334711c; 1.465 GPU-h measured (4 budgeted). Round-6 (base-separator-control) artifacts: eval JSONs (11 GPU-phase fit/anchor + transfer + similarity) under eval_results/issue_825/base-separator-control/ committed at 408a7c3389 + b2c45f639b on branch issue-825 (the instance-local commit c6985f1555 named in the run sentinel never reached a remote; its 11 GPU-phase blobs are content-identical (matching git blob SHAs) to those at 408a7c3389; b2c45f639b later added the transfer + similarity JSONs — record-corrected per upload-verification v8); run code @ 1c284be781 (round scripts; plan v18); HF prefix issue825_base_sep_control @ d4085b09 — analysis_tensors/armC/ (base separator store, 17 files ≈ 2.9 GB, byte-verified), analysis_tensors/preds/ (per-cell held-out prediction npz ×2), analysis_tensors/maps/ (fitted base maps at frozen layers ×4), eval_results_mirror/ (11 JSONs); staged inputs (all read at data-repo revision 79dbfeef; per-artifact fitness: the pair files ARE the control recipe — revision-pinned content identity; the instruct armC store shares single-run provenance with the pairs, upload-date coherence probe PASS; the base chat turnstore is the exact store the committed 0.588 cell was fit from, re-verified by the chat-stream gate reproducing 0.5877 exactly): #931 pair files @ 9534b998 + instruct armC store (issue931_story_map/analysis_tensors/armC/) + the base chat turnstore @ a23e79f17f; round figures @ b5959c2128 (interp-critique r1–r2 revisions: hero transfer bars re-plotted on the full-n chat-ceiling convention matching the body prose @5ea2196e3a, then scatter axis labels shortened after a mid-word clip @1f6db9f0c0); ~2.5 GPU-h measured (2 budgeted, ≤3 scope cap; first spot launch provider-preempted mid-fits). Round-7 (onpolicy-separator-control) artifacts: eval JSONs (25 fit/anchor/decision + 2 Phase-C transfer legs) under eval_results/issue_825/onpolicy-separator-control/ committed at b285f235ee (fits, landed from the HF eval_results_mirror — the run sentinel's pod-local final_commit_sha ea6d9af8 never reached a remote; GCE push gap recorded by upload-verification v9) + 7421328c0d (Phase-C transfer + figures) on branch issue-825; run code @ b3a428d1c0 (gen/pairs/dispatch/figures + the rotated-CI fit extension + the matched-n leg + the transfer-script source generalization; plan v21) with the Phase-C output-routing fix @ fda8f98ba6; HF prefix issue825_onpolicy_sep_control @ 4435ced2 — raw_completions/generation/{base,instruct}/ (continuations + audits + pairs), analysis_tensors/armC_{base,instruct}/ (~2.9 GB stores each) + analysis_tensors/maps/, eval_results_mirror/ (25 JSONs; live-verified this session via scoped list_repo_tree, 75 files); reused inputs re-verified at stage time (pair files @ 9534b998 — fit: the control recipe's own pinned pairs; round-6 base separator store @ d4085b09 + #931 instruct store @ d959b0c6 — fit: the exogenous anchor cells these gates reproduce; chat turnstores @ a23e79f17f — fit: the exact stores the committed ceilings were fit from, re-certified by the chat-stream gates); round figures @ 7421328c0d; 2.907 GPU-h measured (3 budgeted; one GCE flex-start A100-80 launch, finalized after upload-verification PASS v9). Round-7 revision (interp-critique round): position-matched exogenous refit — eval_results/issue_825/onpolicy-separator-control/position_matched_wex_{base,instruct}.json + scripts/issue825_position_matched_wex.py + scripts/issue825_posmatch_figure.py + the refit figure, committed at 5db8b48bd4 on branch issue-825; 0 GPU-h (VM CPU, detached). Interp-critique round 2: 10-seed instruct subsample extension (--extend-subsample mode) — eval_results/issue_825/onpolicy-separator-control/position_matched_wex_instruct_v2.json + scripts/issue825_posmatch_extfig.py + the seed-band figure, committed at ad9a2f047a on branch issue-825; 0 GPU-h (VM CPU).

Context: Created 2026-07-01 from user chat. Verbatim originating prompt:

Experiment: Is the context vector to answer profile mapping present in the base model and does it hold for the user?

Motivation

• We showed that there is a context vector to answer profile mapping (issue 722 I think) for the assistant in the Qwen 2.5 7B instruct model
• We are interested now in seeing:
  • is this mapping present in the base model?
    • how much does it change post instruction -> probably a lot but still good to test
  • does this mapping exist for the user?
    • how much does it change post instruction
    • this user mapping is not super necessary in post training because the model doesn't have to predict the user turn but

Questions

• Does this even make sense because we can't even get the base model to really generate from a chat template?
• Do our mapping results hold

Methodology

• Do the same experiment from issue 722 (linear mapping only -- at all layers) with the following changes:
  • test corresponding user context vector -> user answer profile mapping
  • test in the base model (https://huggingface.co/Qwen/Qwen2.5-7B) -- (make sure this is the proper base model) for both user and assistant
  • also test the same in the finetuned model
• Considerations:
  • we need multi turn conversations to test this -- answer mean should only be taken over the current "speaker's" answer
  • try one with the chat template and one replacing it with more naturalistic format (e.g. semicolon)
  • also plot how well we can predict each token in the answer (separately -- per position) with the context vector
  • also check if you can predict user response from the assistant context vector (and vice versa)
• Reuse as much code as possible

(Quoted text is verbatim; the one model-repo URL inside the quote is hyperlinked to a pinned revision so the footer's URL-permanence gate holds — the displayed text is unchanged.) Follow-up user decisions (same chat): frame on #779's per-context (per-example) map rather than #722's across-context map ("We are interested more in the per-context map"); Track M conversations are generated with Claude Haiku 4.5 as the user; file as proposed only (no autonomous session spawned). Lineage: #779 — the parent single-turn instruct rig this run replicates and extends. Run 2026-07-02 (7 launch attempts; crash-fix rounds 1–6: extract host-RAM OOM → block-wise flush → host upsize → GPU Gram-ridge fit). Interpretation reviewed rounds 1–3 (two critic REVISE rounds, then PASS+PASS). Follow-up round mlp-unprobed-cells run 2026-07-03 (cheap-band auto-run, redundancy-screened; plan v6; 3 launches, results rescued from the crash-persist upload). Follow-up round onpolicy-user-turn run 2026-07-03 (user-requested; verbatim prompt: "run on-policy user turn for both instruct and base model"; plan v7; 4 GCP launches — three extraction crashes on a zero-width-span tokenizer edge in short self-written turns, then a clean run). Follow-up round real-user-turn-null run 2026-07-03 (cheap-band auto-run round 2 of 2, source proposer-9b-cheap; plan v11; one GCP flex-start launch preempted three times by the provider → RunPod failover, one wiring-phase crash on a Unicode line-separator JSONL edge, then a clean run). Inline free analysis crossmodel-map-transfer run 2026-07-03 (user chat; ask: compare the base model's fitted context→answer-profile map to the instruct model's map directly — cross-model transfer R² plus weight-space similarity of the two fitted ridge maps). Follow-up round role-map-comparison run 2026-07-04 (user-requested; verbatim prompt: "Also do a comparison of the user context to behavior mapping with the assistant context to behavior mapping" — clarified in-chat: "i meant context to answer mapping sorry"; plan v14; sequenced after the on-policy and real-conversation rounds so the paired contrast covers all three provenances; tensors-only re-fit, one GCP flex-start launch, clean run). Follow-up round base-separator-control run 2026-07-07 (user-requested; verbatim prompt: "Run it on base too", following "did base model compare against mappings between subsequent punctuation tokens?"; plan v18; two spot-A100-80 launches — the first provider-preempted mid-fits — then a clean run; transfer/similarity/figures computed on the VM). Follow-up round onpolicy-separator-control run 2026-07-07 (user-requested; verbatim prompts: "yes" to the proposed on-policy separator arm, then "do it in both pretrained and instruct model"; plan v21; one GCE flex-start A100-80 launch, clean run; VM Phase C relaunched once after an output-routing fix).