Changing surface framing costs the context-to-answer map far more than changing speaker identity, and changing both adds little beyond framing's cost (LOW confidence)

Methodology: docs/methodology/issue_1689.md · gist

Takeaways

• Into the highest-ceiling target (instruct assistant-chat), full-correction transfers recover median 0.86-0.88 of ceiling for identity-only changes, 0.41-0.47 for framing-only, and 0.28-0.34 for both, so framing, not identity, carries the cost.
• Chat-framing identity swaps reconcile at rung 1 (direct transfer); assistant framing pairs reconcile at no rung in 12 of 12 base and 7 of 12 instruct (pair, arm) combinations; raw operator cosines back the ordering (identity-only 0.14-0.27 vs framing-changing 0.01-0.16).
• Real second user turns are less linearly predictable from the layer-19 prefix than Haiku-simulated ones on the same 3,800 conversations — 6 of 6 framing×model pairs, Δ real − haiku −0.11 to −0.17, mean prefix R² 0.23 vs 0.38 (0.31 vs 0.45 excluding the construct-suspect plain-text cells); length, refusal-template, and corpus-mix confounders unaddressed (Result 11) — a measurement claim, not a mechanism claim. The user-slot recapture round (corrected boundaries) adds the genuine third provenance arm: constant-string fallback 0.63-0.77, Haiku-simulated 0.23-0.34, on-policy Qwen 0.00-0.07 (clean slot to turn-end target, 8 cells), first user turns at the floor (−0.002 to −0.075).
• The one-shared-readout reduction fails on construction, not conditioning: refit in a well-posed reduced basis (n_train at least 2k on every fold), 0 of 18 informative parent rung-1 pair-arms read shared-readout-supported (pooled 34 of 212 against the 50% line, every crossing unit a construct-invalid user arm), so the conjugation construction is falsified and the question passes to a joint constrained transfer-plus-readout fit.
• The fitted context-side transfer maps stay broad-spectrum in both estimator regimes (median correction effective rank 3,489 of 3,584 ambient; 0.80 of available rank well-posed); rank ≤ 128 corrections reconcile at most 14 of 222 deep pair-arms, and all 22 informative base-instruct same-condition pairs read rung 9 (8 reconcile; median recovery across all 22, 0.82).
• Measurement validity + coverage: 54 of 126 context-arm pairs are construct-invalid, 31-52 more per model-arm have non-positive ceilings, and the ceilings are not grid-limited (0 of 84 cell-arm ceilings move by more than 0.02 under a ridge grid widened 3 decades); the parent's "lmsys" user arm was a constant-string fallback and the user-on-policy cells duplicate the haiku cells, so a fifth-follow-up-round real-u2 capture on ~3,800 multi-turn LMSYS+WildChat conversations restored true tier-1 real user data; the fit batteries reached full coverage (588 of 588 units, 1,176 paired ambient-reduced) after a checkpoint-key fix + re-sweep.

Goal

This experiment in context: Prior tasks measured the context-to-answer linear map inside single corpora: #825 fit the map on LMSYS two-turn conversations, #1345 found story-framing transfer fails asymmetrically, and #1310 found character shifts re-encode the answer side, plus a three-cell aligned-cosine contrast where identity-only similarity (0.740) exceeded framing-only (0.488) and both-changed (0.485). Because every prior cell lived in a different corpus, the mapping-similarity ladder's data-paired middle rungs were never computable across speaker or framing. This experiment builds ONE row-paired corpus rendered into 21 speaker-by-framing conditions per model and asks, for 126 ordered condition pairs, which of 9 correction tiers (direct transfer through full A·M·B reparameterization) is the weakest that reconciles the two fitted maps, in Qwen2.5-7B base and instruct, both mapping arms (prefix and context). A user-requested addendum adds the framing-by-identity interaction test on continuous recovery reads, and a real-u2 capture round (2026-08-03) supplies the exact-comparability real second-user-turn arm the parent's constant-string lmsys fallback never provided.

Broader narrative: If "who speaks" only re-coordinates a shared context-to-answer operator, persona is a shallow wrapper over one mechanism; if each speaker or framing carries a genuinely different operator, persona transfer and leakage prediction need per-condition maps. This lattice makes "same operator in different coordinates" vs "different operator" decidable on shared rows — and finds surface framing, not speaker identity, is where the operator changes.

Methodology

Design: N=3,800 two-turn LMSYS conversations (English, exactly 2 turns, token-budget feasible under the tightest framing — chat template plus persona header) rendered into 21 conditions: 7 identities (assistant; user with real-LMSYS / haiku-simulated / self-generated second turns; HELIOS, an AI character; Wren, an assistant-like human; Dana, an ordinary person) crossed with 3 framings (chat template; plain text; narrative story). All non-user completions are generated on-policy by the measured model and judge-filtered; each conversation appears up to 3 times in the generation files and the ladder keeps the first judge-kept row per conversation. Activations are captured teacher-forced at layer 19 (layers 14/18/26 captured but never fit — documented scope shrinkage): X_prefix = end of everything before the second user turn; X_context = one token past the end of the second user turn; Y = end of the answer span. For user cells the answer span IS the second user turn, so X_context equals Y by construction (identity+bias baseline R² = 1.0 exactly) — every user-cell context-arm read is construct-invalid, and in plain-text framing the user-cell prefix boundary also collapses onto the answer end (both arms invalid). Ordered pairs (126 per model): a 40-pair spoke through assistant-chat, within-identity framing pairs, within-framing identity pairs, and user-provenance pairs; both arms fit for every pair. A zero-GPU follow-up round (2026-07-29) added the plan addendum's per-pair Procrustes operator-cosine battery over the same layer-19 store. A second zero-GPU follow-up round (2026-07-29, wider-lambda-ceilings) refit all 84 per-cell layer-19 within-cell ceilings on a ridge grid widened three decades, with a conditional ladder re-read for any pair whose ceiling moved (none did). A third follow-up round (2026-07-29, derived-vs-free-answer-map, user-chat scope from a mentor-posed question) ran three fit batteries over the same layer-19 store pin. First, per ordered pair-arm, a free answer-space map B_free (d²+d parameters) compared against two maps derived from a fitted affine context-space transfer M — B_derived = W_S M W_S⁺ with a derived bias (one shared readout W_S) and B_derived2 = W_T M W_S⁺ (source and target readouts) — verdicted per unit by g1 = R²(B_derived) − 0.9·R²(B_free) and g2 = R²(B_derived2) − 0.9·R²(B_free) into four disjoint classes (shared-readout-supported g1 ≥ 0; readout-changed g1 < 0 and g2 ≥ 0; transfer-map-insufficient otherwise; free-map-uninformative — R²(B_free) below the identity+bias baseline — excluded from counts), with R²(B_derived) the max over truncation ranks 32/128/512/eff-rank and a fixed-eff-rank companion lattice. Second, a structural battery over M (nested class ladder translation → scaled/rotated translation → rank-k → full affine, distance from identity, correction-direction vs diff-of-means alignment, cross-pair top-32 correction-subspace overlap vs a random-subspace null) plus a rank-k reconciliation sweep (k = 1..128) for pairs the parent left at rung ≥ 7. Third, the 21 base-instruct same-condition pairs × 2 directions through the parent 9-rung ladder plus both batteries. Coverage defect and fix: the within-model batteries initially keyed per-unit checkpoints by (pair, arm) without the model, so the second model's unit resumed on the first model's checkpoint (241 base, 11 instruct of 504 computed). A fix round qualified the unit keys by model, migrated the surviving files, and made the re-merged summaries fail loud on duplicate keys; a re-sweep then completed the remaining units — both within-model batteries now report 504 of 504 complete (0 failed, 0 key-mismatched), cross-model unit keys carried both models throughout (84 of 84), and the ≥95%-of-588-units completeness criterion is met (588 of 588). A fourth follow-up round (2026-07-30, wellposed-shared-readout) re-ran all three batteries with every operator fit inside a per-(pair, arm, fold) train-fold-only shared PCA basis of rank k = min(1024, floor of half the smallest train fold) — so n_train ≥ 2k holds on every fold by construction, where the ambient rounds fit at n_train below d = 3,584 — one basis per side shared across source and target, the primary verdict R² read in reduced coordinates with the ambient-reconstruction companion and held-out captured-variance fraction persisted per unit; its one crash-fix round staged the parent per-unit inputs from an HF mirror (the SLURM lane's rsync ships no committed eval_results/), after which the four batteries completed 504 + 504 + 84 + 84 of expected units with the ambient no-op parity gate at zero difference. A fifth follow-up round (2026-08-03, real-u2-capture, source user-chat) captured the exact-comparability real second-user-turn arm the parent's lmsys cell never provided: ~3,800 real multi-turn LMSYS+WildChat conversations (each carrying a real second user turn), rendered into user-identity cells only under three framings × two models, and captured teacher-forced at layer 19 under the parent's user_slot_recapture conventions; a haiku-simulated u2 companion arm was captured on the SAME 3,800 conversations, giving the within-corpus real-vs-simulated contrast the parent's constant-string lmsys arm never supported. Within-cell ridge (reduced-basis rule where n_train < 2k applies), identity+bias baseline, kNN retrieval, and a 40-draw shuffled-answer null were fit per cell; SLURM job 18578 completed 2026-08-04T00:15:46Z. All round counts in this body come from the per-unit JSONs via the digests (analyzer/dvf_unit_digest.csv; well-posed round, analyzer/dvf_wellposed_paired_digest.csv; real-u2 round, eval_results/issue_1689/real_u2_capture/per_cell_summary.json). Conciseness note: the body carries 19 result sections for the 126-pair lattice and its follow-up rounds, so the total-prose-budget WARN, residual per-result word-cap and paragraph sentence-cap WARNs, and over-30-word Takeaways bullet WARNs are acknowledged deliberately, as is the deliberate link (not embed) of the base-model heatmap beside its instruct sibling, and the abbreviated condition labels rendered inside the cross-model scatter (space constraints; the conditions are defined in plain English under Design), and the two driver-rendered well-posed round figures ship without plot sidecars (their per-unit data is the committed paired digest).

Training: N/A — no model training. Capture/fit/generation hyperparameters:

Hyperparameter	Value	Source
Models	Qwen/Qwen2.5-7B, Qwen/Qwen2.5-7B-Instruct	plan §4
Corpus	3,800 LMSYS two-turn conversations (oversampled from the 3,000 target)	scripts/issue1689_common.py N_SOURCE_LMSYS
Layer fit	19 (of captured 14/18/19/26)	epm:results card, v75 descope
Ridge λ grid	np.logspace(-2, 4, 13), inner 3-fold conversation-grouped CV	issue1689_fit_ladder.py LAMBDAS, _fit_ridge_inner_group_cv
Pair split	single conversation-grouped 80/20 split (fold 0 of 5 as test); source map fit on all source rows	issue1689_fit_ladder.py _ladder_pair_core (deviation from plan §4's per-rung 5-fold CV)
Reconciliation bar	transfer R² ≥ 0.90 × within-target held-out R²; recomputed at 0.85 / 0.95	issue1689_common.py RUNG_REACHED_THRESHOLD; plan §6.6 item 3
Bootstrap	200 conversation-level draws on 76 pairs (39 base, 37 instruct) checkpointed before the final descope; 0 draws on the rest	ladder JSONs bootstrap_draws.n_draws; plan 1000 → 200 (R15) → 0 (v75, user-directed)
Null	40-draw matched-capacity shuffled-answer null, every pair	ladder JSONs; plan §5
Generation	temperature 0.7, top-p 0.95, max_new_tokens 1024, n=1 per prompt, vLLM	issue1689_common.py GEN_TEMPERATURE/GEN_TOP_P/GEN_MAX_NEW_TOKENS
Judge	claude-sonnet-4-5-20250929, 3 draws, temperature 0.7, max_tokens 300, keep mean ≥ 50	issue1689_common.py JUDGE_*
Yield floor	0.80 (missed by all 30 pools)	issue1689_common.py YIELD_FLOOR; onpolicy_stats/
Fit engine	torch fp64 GPU, numpy-equivalence gated at atol 1e-4 (PASS, 4 of 4 pair-arms)	epm:results card
Procrustes battery (follow-up round)	per-cell ridge on all cell rows, same λ grid; data-paired orthogonal alignment on each pair's conversation intersection; 200 Haar rotation-null draws, seed 42	issue1689_procrustes_battery.py summary_L19.json metadata
Wider-λ re-check (follow-up round)	grid np.logspace(-2, 7, 19) (superset of the published 13-point grid, same half-decade spacing), same inner 3-fold conversation-grouped CV, 5 outer folds, seed 42; parity gate: a 13-grid refit of the largest cell-arm must reproduce the published R² within 1e-3 (measured gap 2.2e-16); store at HF revision d1010a25f8	issue1689_lambda_recheck.py; wider-lambda-ceilings/summary.json
Derived-vs-free round (follow-up)	ladder13 λ grid; 5 conversation-grouped outer folds, seed 42; truncation ranks 32/128/512/eff-rank (verdict = max over ranks, fixed-eff-rank companion); rank sweep k ∈ 1, 2, 4, ..., 128; 200-draw rotation nulls; 40-draw class/rank matched-capacity nulls; all derived quantities from train-fold statistics only; parity gate vs the published parent pair: max abs rung-R² diff 0.000e+00; store at HF revision d1010a25f8	issue1689_derived_vs_free.py, issue1689_context_map_structure.py; derived_vs_free_B/gate1_report.json
Well-posed re-fit round (follow-up)	per-unit shared PCA basis, train-fold-only, rank k = min(1024, floor(min-fold n_train/2)), one basis per side shared across source and target; same ladder13 grid, 5 folds, seed 42; truncation ranks 32/128/512/eff-rank clamped to [1, k]; rotation and matched-capacity nulls drawn in dimension k; Gate-1 ambient no-op parity max abs scalar diff 0.0	issue1689_derived_vs_free.py --fit-basis reduced; derived_vs_free_wellposed/gate1_report.json + summary.json meta
Real-u2 capture round (follow-up)	3,800 real multi-turn conversations sampled from LMSYS/WildChat (≥2 user turns; #1738 eligibility machinery + near-dupe gating); user-identity cells only, 3 framings × 2 models × 2 provenances (real u2 + haiku-simulated u2 on the same 3,800 conversations); teacher-forced L19 capture at user_slot_recapture X_prefix / X_context / Y = second-user-turn end; within-cell ridge, ladder13 λ grid, 5 conversation-grouped folds, 40-draw shuffled-answer null, identity+bias baseline, kNN retrieval; SLURM job 18578, completed 2026-08-04T00:15:46Z	scripts/issue1689_realu2_*.py; eval_results/issue_1689/real_u2_capture/per_cell_summary.json + rung_reached_matrix.json
User-slot recapture round (follow-up)	corrected token boundaries — clean read slot = last token strictly before the predicted turn (chat: user-header end; plain text: speaker-label end; story: the turn’s opening quotation mark), with the parent’s straddle slot kept as a comparison read; 24 user-identity cells (2 models × 4 framings × 3 provenances) plus single-turn, parent-recap, and on-policy-a1 bridge cells; story cells carry no first-turn, floor-control, or parent-parity read (no chat header exists there); parent-parity rows exist only for the two chat constant-string cells; the round’s defect record logs the parent’s plain-text user-cell boundary collapse (prefix, context, and answer ends coincided; identity+bias R² exactly 1.0), the stale unfilled-placeholder u2 embedded in parent haiku/on-policy prompts, and the constant-string lmsys fallback (not fixable from persisted data; carried forward labelled); ladder13 λ grid, 5 conversation-grouped folds, seed 42, L19; per-cell values are point estimates (fold spread not persisted in the mirror); gate1 parent parity PASS at tol 0.01, abs Δ 2.4e-4 base / 5.8e-4 instruct (max 5.8e-4); SLURM job 16221, completed 2026-07-31	eval_results/issue_1689/user_slot_recapture/summary.json (HF mirror issue1689_speaker_lattice/user_slot_recapture/eval_mirror/)

Evaluation: For each ordered pair (source S, target T) and arm, the source ridge map is corrected by 9 successively stronger tiers — 1 direct, 2 context offset, 3 answer offset, 4 bias refit, 5 global scale, 6 orthogonal rotation, 7 context-side ridge reparameterization (A-only), 8 answer-side (B-only), 9 full A·M·B — each read as held-out R² on T against the reconciliation bar. The fit script codes "no rung reconciles" as rung 9; this analysis separates the two (an explicit no-reconcile code) and adds recovery fractions (rung R² divided by the target's within-cell ceiling). The matched-capacity null is degenerate for the rung statistic: shuffling answers drives the target ceiling non-positive, the bar becomes vacuous, and every null draw reads rung 1 (null p97.5 = 1.0 on 504 of 504 pair-arms) — a rung-1 observation is therefore not separable from the null by the ordinal alone, and discrimination rests on the ceilings and recovery fractions. Identity+learned-bias baseline and kNN retrieval (k in 1/5/10, euclidean and cosine, chance stated per pool) are reported per cell in percell/; the identity+bias baseline is what exposes the degenerate user arms. Robustness check (resolved by the wider-grid round): the inner-CV ridge λ selection pinned at the grid ceiling (λ = 10^4) in 265 of 420 layer-19 fold-fits; refitting all 84 cell-arms with the ceiling raised to 10^7 moved no ceiling by more than 0.02 and left no fold-fit selection above 10^5, so neither the ceilings nor the rung reads that inherit them are grid-limited (see the wider-grid result). No p-values are computed by the rig; uncertainty reads come from the 200-draw conversation-level bootstrap where present and from cross-pair spread elsewhere. Language-intrusion audit (Qwen under an English eval): CJK-matching rows are 0.4-2.2% of kept completions in the assistant and instruct pools the headline reads rest on, and 6.7-9.4% in base character-chat and base user-on-policy pools (e.g. 50 of 531 HELIOS-chat rows, 34 of 480 user-on-policy-chat rows); no judged-rate headline exists to recount, and the affected pools are already flagged as low-yield. Derived-vs-free round reading conventions: every R² in the round is n_train < d = 3,584 shrunk ridge, read comparatively across map forms within a pair, never as an absolute ceiling; the weakest-class and minimal-k reads are conditional on that estimator (lower-bound-style, not intrinsic-rank claims); the matched-capacity class/rank nulls inherit the parent null's degeneracy (shuffled answers collapse the bar, so a translation-sufficient or low-k observation is not separable from the null by the ordinal alone); the cross-model reads are scoped to model-specific on-policy answer distributions (each side of a cross-model pair carries its own model's answer text). The well-posed re-fit round removes the n_train below d caveat for its own reads: its verdict R² live in per-unit reduced coordinates, conditional on the captured subspace (held-out captured-variance medians 0.30-0.92 across k-bands), and its reduced-coordinate identity+bias read is basis-relative — used as a class-0 screen only, never compared against the ambient identity baseline. Real-u2 capture round reading conventions: within-cell ridge R² is read against the 40-draw matched-capacity null's 97.5th percentile (~-0.03 nats across cells; every observed prefix R² exceeds it by ≥0.10, so the maps are learning) and against the identity+bias baseline (deeply negative on the prefix arm, so the map is not a trivial coordinate shift). Two of six framings — naturalistic (plain-text) on both models — have context-arm construct invalidity because the user_slot_recapture prefix boundary collapses onto the answer end there (context R² ≥ 0.93, identity+bias-driven); treat those context reads as self-prediction, not operator learning, and rely on the prefix arm.

Real-u2 round: unaddressed confounders (Result 11 alternative explanations, cap-excluded here): three structural confounders sit between the observed prefix-R² gap and any generator-side mechanism claim, each rule-out-able as a follow-up read on the persisted round artifacts. First, length-tail confound. Real u2 lengths (real_multiturn_first_exchange.jsonl at HF revision 068694e11b) are heavy-tailed — mean 316 chars, median 63, p10 16, p90 614, max 27,144 — while Haiku u2 lengths (haiku_u2.jsonl, same revision) are unimodal-narrow — mean 168, median 139, p10 74, p90 285, max 1,107; the two distributions barely overlap in shape (structural scan over all 3,800 conversations). A ridge fit onto a more-concentrated target reports higher held-out R² almost by definition, so length matching (per-conversation subsampling to matched-length pairs, or per-length-decile R²) would carry a substantial fraction of the Δ before any generator property is invoked. Second, refusal-template artifact. 177 of 3,800 haiku u2 rows (4.66%) match a refusal-template regex — I can[' ]?t (continue|help|write|generate|do|provide|assist|do this|proceed|comply)|I'm not able to|I won't|I cannot|violates my|usage polic|I cannot fulfill|I am unable — over the harmful/adult-content LMSYS narratives; these refusal rows are stereotyped, fixed-template targets that a linear map recovers more easily than the raw underlying user text (refusal-row length mean 491 chars, median 472, vs the haiku-arm median 139 chars overall — refusal rows are 3× longer than the median haiku reply while replacing very-short real u2 turns). Rule out: recompute the Δ on the haiku-non-refusal subset (≈3,623 rows) against the matched conv_ids in real u2. Third, bimodal corpus mix. The corpus is 1,923 LMSYS + 1,877 WildChat rows (conv_id positional match 3,800/3,800 across provenances, so the pairing is symmetric); the LMSYS half is dominated by narrative "continue" bumpers and NAME-scrubbed continuations, the WildChat half by follow-up Q&A. The aggregate real-u2 distribution is bimodal over two structurally different sub-corpora and the linear map fits a mixture. Rule out: per-corpus split of the same reads (LMSYS-only vs WildChat-only R²). Related internal inconsistency: this fold's Methodology (paragraph above) flags plain-text framing as user-prefix boundary-collapse construct-suspect on BOTH arms; the JSON's prefix_arm.construct_invalid flag reads False on those two cells (only the context arm carries the JSON flag) — the n=4 exclude-naturalistic mean in the Result 11 takeaway is the discipline-consistent read, the n=6 mean is quoted for full-cell transparency; a durable fix reconciles either the flag or the caveat.

Data extraction: Ladder outputs: eval_results/issue_1689/ladder/ladder_<model>_L19.json (126 pairs × 2 arms: 9 point R²s, within-target ceiling, reach bar, rung-reached, 40-draw null, bootstrap draws where present) plus per-pair checkpoints in pairs_<model>_L19/. Per-cell ceilings and baselines: eval_results/issue_1689/percell/ (42 JSONs). Generation yields and judge drop classes: eval_results/issue_1689/onpolicy_stats/ (30 JSONs; the categorized drop counters overlap and do not reconcile with n_input minus kept, so only yield_frac — verified equal to the kept-store row counts — is quoted). The analyzer digest (validity flags, recovery fractions, threshold sweep) is eval_results/issue_1689/analyzer/pair_digest.csv, produced by scripts/issue1689_analyzer_digest.py; figures by scripts/issue1689_analyzer_figures.py. Follow-up battery: eval_results/issue_1689/procrustes/ (two per-model battery JSONs + summary_L19.json) from scripts/issue1689_procrustes_battery.py — per-cell maps refit on all cell rows, a fit-grain deviation from the ladder's per-pair paired-row source fits; alignment uses each pair's conversation intersection (96-3,800 shared rows, below d = 3,584, so the rotation beyond the informed rank is an arbitrary orthogonal completion); figure by scripts/issue1689_procrustes_figure.py. Wider-λ re-check: eval_results/issue_1689/wider-lambda-ceilings/ (summary.json, an empty affected_pairs.json, percell_wide19/ — 84 per-cell-arm JSONs with per-fold selected λs and the parity fields) from scripts/issue1689_lambda_recheck.py; figure by scripts/issue1689_lambda_recheck_figure.py. Derived-vs-free round: eval_results/issue_1689/derived_vs_free_B/ (504 model-qualified per-unit JSONs, gate1_report.json), context_map_structure/ (504 per-unit JSONs, subspace_overlap.json), crossmodel_pairs/ (84 ladder units in ladder_crossmodel_L19.json + per-pair checkpoints, 84 derived-vs-free units, the cross-model structural battery + its subspace_overlap.json); the re-merged summary.json files are model-qualified and assert against duplicate keys (see Design); body counts come from the per-unit digest analyzer/dvf_unit_digest.csv from scripts/issue1689_dvf_fold_digest.py; fold figures by scripts/issue1689_dvf_fold_figures.py. Well-posed re-fit round: eval_results/issue_1689/derived_vs_free_wellposed/ (504 per-unit JSONs, summary.json, gate1_report.json, pilot_pair.json, fence_report.json), context_map_structure_wellposed/ (504 units), crossmodel_pairs_wellposed/ (84 units, plus the nested crossmodel_structure_wellposed/, 84 units); the paired ambient-vs-reduced digest analyzer/dvf_wellposed_paired_digest.csv (1,176 rows, zero unmatched keys) and its aggregate dvf_wellposed_paired_summary.json come from the extended scripts/issue1689_dvf_fold_digest.py; round figures by the extended scripts/issue1689_derived_vs_free_figures.py. Real-u2 capture round: eval_results/issue_1689/real_u2_capture/per_cell_summary.json (12 cells: prefix arm + context arm, r2_obs, r2_null_mean/p025/p975, identity_bias_r2, kNN retrieval per arm, lambda_selected_per_fold, construct_invalid flag on the naturalistic context arms) + rung_reached_matrix.json (compact per-cell headline) from the round's driver + fit scripts committed at fits SHA 376200beb71839c7583168cda0223629b71fd565. Round figures fig14_real_vs_haiku_prefix_r2 (hero) and fig15_real_u2_r2_lattice (per-cell lattice) from scripts/issue1689_real_u2_figures.py at commit e1d6e27d003c81ba27db36fb9bb967a929480ad8. Compact SVD factor bundles on the HF data repo at derived_vs_free/analysis_tensors — 1,681 files counted live via list_repo_tree: the current 504+504 within-model plus 84+84 cross-model unit bundles, beside 252+252 superseded pre-fix names and one upload probe.

Sample training/evaluation data + completions: Random sample (seed 42) of 3 of 8,484 kept rows, sanitized for context hygiene (~15-word excerpts; LMSYS-class real-user corpus). Full rows: assistant-chat instruct shards.

row 1309 (judge 95.0): u1 "The subject is entity that performs the action of the verb..." ->
  a2 "Certainly! Here is a more detailed breakdown of the information extracted from the news:" [truncated]
row 228 (judge 95.0): u1 "Write a fizzbuzz program in python" ->
  a2 "Sure! Let's break down the FizzBuzz program in Python and explain it step by step." [truncated]
row 51 (judge 95.0): u1 "help me to explain Theoretical solution for tubular steel sections" ->
  a2 "Certainly! When analyzing tubular steel sections, we can use theoretical methods and Finit" [truncated]

Random sample (seed 42) of 2 of 480 kept user-on-policy rows, same sanitization; full rows: user-on-policy chat base file. Both self-generated "user" turns read like assistant text — which is why 96% of this pool failed the user-likeness judge (and this arm's capture was defective regardless; see the duplication result):

row 327 (judge 55.0): u1 "I want to create a multimodal AI model. you help me?..." ->
  u2 "Sure, you can use a pre-trained language model like GPT-3 to generate the architectural pl" [truncated]
row 57 (judge 78.3): u1 "what are some best practices for time management?" ->
  u2 "Sure, here's a list of some of the top 100 best practices for time management: 1. Priorit" [truncated]

Judge-dropped rows were not persisted (only kept rows ship under raw_completions/gen/), so dropped (non-firing) examples cannot be quoted — a persistence gap of the run, not of this analysis.

Random sample (seed 42) of 3 of 3,800 conv_id-matched (real u2, haiku u2) row pairs from the real-u2 capture round, sanitized for context hygiene (LMSYS/WildChat unscreened real-user corpus; ≤15-word excerpts, harmful-content-corpus rows abstracted): one refusal-template haiku pair to expose the confounder, two benign pairs (a WildChat Q&A, a short-turn LMSYS pair). Full rows: real u2 corpus · haiku u2 file.

row 0 (LMSYS; haiku_u2 is a refusal template; sanitized):
  u1: [adult-narrative LMSYS prompt — sanitized; 449 words; verify at raw path]
  real_u2 (21 words / 123 chars): "And then they are suddenly approached by NAME_2 ..." [truncated — harmful-content-corpus row; verbatim at HF raw path]
  haiku_u2 (81 words / 499 chars, refusal template): "I can't continue this story. The narrative describes [content that triggers a refusal-template response] ..." [truncated — harmful-content-corpus row; verbatim at HF raw path]

row 242 (WildChat; benign; Q&A follow-up):
  u1: "You are an experienced Canadian art educator and creative writer of academic nonfiction with the ..." [truncated]
  real_u2 (10 words / 61 chars): "Try again, this time focusing on concepts rather than authors"
  haiku_u2 (49 words / 343 chars): "I like this structure, but I'm concerned the subheadings still reference the theorists too prominently—can ..." [truncated]

row 1368 (LMSYS; benign; short-turn):
  u1 (10 words): "can you solve this equation x^2 + 3.0x = 0?"
  real_u2 (14 words / 66 chars): "what are the best ways to comfort who is going through a break up?"
  haiku_u2 (13 words / 58 chars): "wait, why not? can't I factor out x and solve it that way?"

Three structural facts read directly off the sanitized rows (and confirmed by an independent scan of all 3,800 pairs at the HF revision above): first, refusal templates run 3× longer than the real u2 they replace at the harmful-content tail (row 0: 81 vs 21 words) and 177 of 3,800 haiku rows (4.66%) match the refusal-template regex; second, real u2 lengths are heavy-tailed (median 63 chars, p10 16, p90 614) while haiku u2 lengths are unimodal-narrow (median 139 chars, p10 74, p90 285) — the two distributions barely overlap in shape; third, the corpus mixes 1,923 LMSYS + 1,877 WildChat rows on both provenances by design (conv_id positional match 3,800 / 3,800). All three are candidate confounders for Result 11's R² gap; a follow-up read on length-matched pairs and on the haiku-non-refusal subset is queued.

Results

Per-cell ceilings and construct validity leave most of the lattice uninterpretable

Within-cell held-out R² at layer 19 for all 21 conditions × 2 models × 2 arms, from the per-cell fits; hatched red bars mark arms where the identity+bias baseline reads R² = 1.0 exactly.

Figure. Most non-user cells have near-zero ceilings; user arms are degenerate self-predictions. Within-cell held-out R² (L19), prefix (blue) and context (orange) arms per condition; top base, bottom instruct. Hatched red = degenerate arm (user cells: the context span ends where the answer span ends). Value labels per bar; kept rows per cell range 288-11,400.

User-cell context arms (and plain-text user prefix arms) are self-predictions (the predicted user turn is already part of the conditioning context; the two spans end at the same token) — ceilings 0.92-0.99 on the full 3,800-conversation cells, 0.54-0.87 on the low-yield user-on-policy cells, kNN retrieval near-ceiling (median rank exactly 2 in pools of 288-11,400 rows; acc@5 = 0.98-1.00; acc@1 = 0, driven by a same-conversation duplicate row at rank 1 on all 18 arms). They carry no operator information.

Non-user ceilings are low: base 0.00-0.09, instruct 0.01-0.32 (assistant-chat highest). Character-chat cells keep only 417-624 of 11,400 rows after judge filtering; the user-on-policy chat cells keep as few as 288. Every downstream rung read is conditioned on this ceiling heterogeneity.

The bimodal rung distribution mostly reflects ceiling validity

Rung-reached at the 0.90 bar for all 126 ordered pairs, both arms, with validity overlays: gray d = target ceiling at or below zero (the bar is vacuous, rung 1 trivial), light x = construct-invalid arm, dark red = no rung reconciles.

Figure. Rung-1 and rung-9 modes track validity flags, not operator similarity. Instruct model, L19: weakest reconciling rung (viridis 1-9) per ordered pair, prefix (left) and context (right). Base-model version: base heatmap (deliberate link — the two figures are the same read on the sibling model).

The raw rung distribution is bimodal (base context: 72 of 126 pairs at rung 1, 41 at rung 9), but 54 context-arm pairs are construct-invalid and 31-52 per model-arm are ceiling-degenerate; zero base-context rung-1 pairs survive the validity screen, and most rung-9 codes mean no rung reconciles rather than reconciliation at rung 9 (36 of 41, base context; the two outcomes share code 9, separated here by recovery).

Across bars 0.85/0.90/0.95, few pairs change rung: 1 of 28 informative pairs (base context), 2 of 50 (base prefix), 3 of 41 (instruct context), and 10 of 65 (instruct prefix).

Arm parity: 76% of the 126 pairs (base) and 82% (instruct) agree within 1 rung across prefix/context. Base-instruct parity: 83-86% within 1 rung; both figures are inflated by shared degeneracies.

Framing changes need much deeper correction than identity changes

Rung-reached distribution by pair class, informative pairs only (construct-invalid arms and degenerate-ceiling targets excluded).

Figure. Identity pairs reconcile shallowly; framing pairs mostly reconcile at no rung. Informative ordered pairs per class, colored by weakest reconciling rung (viridis bins; dark red = none); panels are model × arm; counts atop bars.

Within-framing identity pairs reconcile shallowly: all 7 informative instruct chat-framing pairs sit at rung 1. On the 6 with target ceiling of at least 0.05, direct transfer recovers 1.2-2.9 times the target's own ceiling (likely because the assistant map is fit on 20 times more rows); the seventh, Wren to Dana, recovers 12.0 times a near-zero 0.005 prefix ceiling. Bootstrap rung ranges stay at 1 on two of the four context pairs with draws and reach rung 3 on the other two; per-draw recovery for the Dana target (context arm) spans 0.03 to 1.32 — small-cell resamples swing widely even where the point estimate stays at rung 1.

Within-identity framing pairs reconcile at no rung in 25 of 30 informative cases. Both pre-stated directional hypotheses fail: framing was predicted shallow and identity deep; the data show the reverse. Identity claims carry the selection caveat above (chat-character cells keep 4-6% of generations).

Assistant framing pairs reconcile only at full reparameterization, and usually not at all

Per-rung recovery curves for the 6 ordered assistant framing pairs (context arm), the per-pair view behind the framing half of the class summary.

Figure. Base framing transfers reach the bar at no rung; instruct only at rung 9, on 3 of 6 pairs. Recovery fraction per rung, assistant framing pairs, context arm; base (left), instruct (right); dashed = the 0.90 bar; shared conversations per pair 1,519-2,317.

Story-to-chat transfers onto the highest-ceiling framing targets recover only 0.28-0.45 of ceiling at the strongest correction (95% bootstrap range 0.26-0.47 base, 0.26-0.36 instruct); chat-to-plain reaches 0.64-0.91 (range 0.38-0.92 across the four draw-bearing directions).

Across both arms, 12 of 12 base and 7 of 12 instruct pair-arms reconcile at no rung. The 5 instruct reconciliations all need rung 9: four land on story targets with ceilings of 0.013-0.019, where recoveries above 1.0 reflect the near-zero ceiling rather than good transfer, and one is marginal — chat-to-plain, context arm, 0.0693 against a 0.0687 bar. On the continuous reads (per-rung recovery fractions and their bootstrap ranges), framing remains the axis needing the deepest correction.

Crossed transfers add little beyond framing's cost

Rung-9 recovery fraction for every transfer into the assistant-chat cell (instruct), grouped by whether the source differs in identity only, framing only, or both; user-source pairs excluded (their answer construct is the user turn, not an assistant answer).

Figure. Crossed transfers land with framing-only transfers, far below identity-only. Rung-9 recovery of the assistant-chat ceiling (instruct, L19), context (left) and prefix (right) arms; labeled points = source cells; thick ticks = group medians; dashed = full recovery.

Descriptive medians over 3 / 2 / 6 source cells per group per arm (too few cells for group CIs): identity-only 0.86 (context) / 0.88 (prefix); framing-only 0.41 / 0.47; both-changed 0.28 / 0.34. The crossed transfers land nearest the framing-only band, about 0.1 below it and roughly 0.5 below identity-only. At this cell count the medians fit both a framing-dominates and an additive-cost account (additive predicts 0.27 / 0.35); the data rule out identity costing anything comparable to framing.

The ordinal pattern matches a prior row-paired three-cell contrast (aligned-cosine, different corpus: identity-only 0.740, framing-only 0.488, both 0.485; not directly comparable). Small identity-source cells bias against identity transfers, so the ordering is conservative there; the judge-selection caveat applies.

Corrections localize to the answer side, for framing pairs too

Context-side-only (rung 7) versus answer-side-only (rung 8) recovery for every informative pair with target ceiling of at least 0.05, both arms pooled.

Figure. Answer-side correction beats context-side for nearly every pair class. Rung-7 (x) vs rung-8 (y) recovery fractions, clipped to plus/minus 1.5; base (left), instruct (right); marker = pair class; dashed diagonal = equal recovery.

The pre-stated prior — framing moves the context side, identity moves the answer side — is not supported: most pairs of every class sit above the diagonal, framing pairs included (instruct assistant chat-to-story: context-side recovery −2.7 to −3.3, answer-side 0.50 to 0.62). Only base chat-vs-plain assistant pairs show a mild context-side edge (0.40-0.56 vs 0.29-0.35).

One alternative stays open: the context-side ridge map must be learned from 1,300-2,300 training conversations in 3,584 dimensions, so its failure may be estimation-limited.

Real and simulated user turns differ by an answer-side transform in the parent grid

Per-rung recovery for the four parent-corpus real-LMSYS vs haiku user-provenance pairs per model, prefix arm — the only construct-valid arm; the prefix text is identical across provenances in the parent's grid by design, so all differences read as answer-side by construction.

Figure. Real-vs-simulated user turns reconcile only at rotation or deeper. Recovery fraction per rung, LMSYS-vs-haiku pairs (chat and story framings, both directions), n=3,800 shared conversations each; base (left), instruct (right); dashed = the 0.90 bar.

These are the highest-ceiling reads in the parent's design (0.41-0.78 on 3,800 conversations). Direct transfer fails (recovery −1.2 to −3.9); the answer-mean offset recovers only 0.20-0.34; rotation recovers 0.73-1.10 and answer-side-only reparameterization 0.59-1.23 (values above 1.0 mean the corrected transfer beats the target's own held-out fit).

Simulated and real user turns are encoded in different answer-side coordinates related by roughly an orthogonal transform. The provenance-shallowness hypothesis (reconciliation at rung 4 or weaker) fails.

Data-realism caveat (corrected): the parent's lmsys arm is a CONSTANT-STRING fallback rather than a true real second user turn, and the "haiku" arm is tier-3 LLM-simulated. Neither is a substitute for real user data on these reads; the fifth-follow-up-round real-u2 capture below supplies the exact-comparability real vs simulated contrast this figure could not.

The user-on-policy arm duplicates the haiku arm at or before capture

Forward-direction versus reverse-direction rung R² for every user-provenance pair (prefix arm, all framings and models): a pair whose two cells hold identical data lands exactly on the diagonal in both directions.

Figure. Haiku and on-policy cells hold identical stored data. Rung R² forward vs reverse per provenance pair family (54 rung values each, clipped to plus/minus 1.2). Haiku-vs-on-policy points sit exactly on the diagonal; LMSYS families scatter off it.

Haiku-vs-on-policy ladder outputs are identical to machine precision in both directions for all 6 framing-model combinations (base chat: ceiling 0.1250491398379796 and every rung R² equal under both orderings), which is only consistent with the two cells' stored activations being identical on shared conversations. Yet the raw on-policy turns differ from haiku's on 160 of 160 shared conversations (SHA-256 text hashes), and the rendered pre-fill rows differ on 3,800 of 3,800. The on-policy turns were generated, but the data was duplicated at or before capture; the root-cause audit is a follow-up.

Every user-on-policy cell is a haiku-arm duplicate on a judge-selected subset; the three-way provenance question collapses to the two-way real-vs-haiku contrast above. Re-capturing the six cells from the persisted on-policy generations would restore the third arm.

Real user turns are less linearly predictable from the prefix than Haiku-simulated ones on matched conversations

Prefix-arm held-out R² per framing, real u2 vs Haiku u2, both captured on the same 3,800 LMSYS+WildChat conversations.

Figure. Real u2 is systematically less linearly predictable than Haiku u2 on matched conversations. Prefix-arm held-out R² (L19) per framing, Qwen 2.5-7B base (left) / instruct (right); real LMSYS/WildChat u2 (blue) vs Haiku-simulated u2 (orange), same 3,800 conversations both provenances. Plain-text framing is construct-suspect per this fold's Methodology (see § Real-u2 round: unaddressed confounders).

All 12 cells learn (prefix R² 0.06-0.52 vs shuffled null p97.5 ≈ −0.03). Paired 6/6 same-direction, Δ real−haiku = −0.11 to −0.17. Mean prefix R² real 0.23 vs haiku 0.38 across all 6 cells; across the 4 non-naturalistic cells (plain-text prefix is construct-suspect in this fold), real 0.31 vs haiku 0.45 (Δ −0.16 to −0.11, still 4/4). The construct-valid context arm (4 cells) matches at smaller magnitude: real 0.45 vs haiku 0.56 (Δ −0.14 to −0.07). Complements Result 6 with an absolute-difficulty read; settles the deferred tier-1-real-data claim.

Three unaddressed confounders can carry a large fraction of the gap — a length-tail confound (real heavy-tailed vs haiku unimodal-narrow), a refusal artifact (177/3,800 = 4.66% haiku rows), and a bimodal corpus mix (LMSYS + WildChat) — all rule-out-able (see ## Methodology § Real-u2). Direction confidence MODERATE (6/6 sign at n=3,800/cell); mechanism confidence LOW until length and refusal rule-outs land.

Per-cell lattice: both arms across the six real-u2 capture cells

The low-level per-cell view behind the hero: both mapping arms per unit, across the 12 real-u2 capture cells (3 framings × 2 models × 2 provenances).

Figure. Per-cell R² lattice, both arms, real-u2 capture round. Prefix arm (blue) and context arm (green) per cell, across 3 framings × 2 provenances × 2 models. Hatched red context bars mark plain-text framing units where the user prefix boundary collapses onto the answer end (self-prediction; treat as construct-invalid). Value labels per bar; n=3,800 conversations per cell.

Two facts read off the lattice. First, the four construct-valid context arms (chat and story framings, both provenances, both models) confirm the hero's paired-Δ direction: haiku always sits above real (context R² haiku 0.44/0.55/0.76/0.47 vs real 0.35/0.41/0.69/0.34). Second, the naturalistic (plain-text) framing renders both provenances' context arms at R² ≥ 0.93 with identity+bias baselines saturating there — the boundary-collapse construct invalidity the parent's user-cell context-arm caveat already documented, now reproduced in this fifth-follow-up-round rig. So the round's operator read rests on the prefix arm across all six framing×provenance conditions per model, and on the four construct-valid framing×provenance context arms for the answer-side companion.

Operator cosines confirm the ordering in raw coordinates; the aligned read tracks shared-row count

Raw operator cosine (per-cell maps compared in shared coordinates, no alignment) against the data-paired Procrustes-aligned cosine, for 87 valid pair-arms per model (39 construct-invalid user arms excluded); per-cell maps refit on all cell rows; both arms pooled.

Figure. Identity pairs lead unaligned; alignment lifts framing pairs and rises with shared rows. Raw (x) vs aligned (y) operator cosine per valid pair-arm, base (left) and instruct (right); color and marker = pair class; labels on aligned of at least 0.50; gray band at zero = rotation-null p97.5 (about 0.0005).

On the assistant-chat spoke, every identity-only pair's raw cosine (0.14-0.27) exceeds every framing-changing pair's (0.01-0.16) in every model and arm — a direction-aware confirmation of the rung ordering. Alignment lifts framing pairs (raw 0.04-0.13 to 0.19-0.29, instruct context) and slightly lowers identity pairs (0.26-0.27 to 0.23), so the three-cell aligned ordering quoted above does not reproduce; aligned cosine instead tracks shared-row count (Spearman 0.56-0.76 over 87 pair-arms; identity-chat intersections hold 114-159 rows vs 2,200-2,317 for framing), an estimation confound.

All pair-arms exceed the rotation-null p97.5, a descriptive read: the Haar null prices random rotations, not the data-fitted alignment, and every pair shares one conversation bank. Haiku vs self-generated user pairs read aligned equal to raw at machine precision, as the duplicated rows above force.

A ridge grid widened three decades moves no ceiling: the published reads are not λ-grid-limited

Change in within-cell held-out R² at layer 19 when the ridge-λ grid extends from logspace(-2, 4, 13) to logspace(-2, 7, 19), per cell-arm (84 = 21 conditions × 2 models × 2 arms), against the published ceiling with the 0.02 mover bar; beside it, fold-level λ-selection counts on both grids.

Figure. No cell-arm moves past the 0.02 bar; λ selections spread only two half-decade steps beyond the old edge. Left: ceiling change vs published ceiling, prefix (blue) and context (orange), circles base / squares instruct; dashed = the 0.02 mover bar. Right: fold-fit λ-selection counts, published 13-point vs wide 19-point grid; dashed = old ceiling, dotted = new.

No cell-arm moves by more than 0.02 (0 of 84, where 5 or more movers would have triggered the conditional ladder re-read; largest +0.019, base-model Dana-story context), so the published reads stand and the affected-pair set is empty. Of the 265 fold-fits pinned at the old 10^4 edge, 202 move up — 100 to 10^4.5 and 102 to 10^5 — and none of the 420 selects above 10^5, three decades below the new edge (zero at 10^7). The pile-up was a λ-selection edge artifact on a flat R² plateau, not a ceiling limitation; the largest changes sit in near-zero-ceiling base-model story cells.

The derived shared-readout map fails its own calibration, leaving the one-readout reduction unresolved

Verdict counts over all 504 derived-vs-free units (four classes, per model and arm, all vs informative), beside per-unit g1 — the derived-minus-0.9×free held-out R² gap — against the parent ladder rung.

Figure. Transfer-map-insufficient dominates in both models, and the gap deepens with parent rung. Left: stacked verdict counts per model and arm, all units and the informative subset. Right: g1 vs parent rung, circles prefix / squares context, colored by verdict; concordance annotated.

On informative units the shared-readout construction never wins in either model: 0 shared-readout-supported, 7 readout-changed, 169 transfer-map-insufficient of 176 counted. The battery fails its calibration bar — 13 of 256 parent rung-1 pair-arms (5%; 0 of 26 informative) read shared-readout-supported against the 50% falsification line — first read as estimation-limited conjugation under shrunk ridge; the well-posed re-fit below refutes that account. Comparative reads replicate across models: Spearman of g1 against parent rung is −0.73/−0.61 in base and −0.68/−0.71 in instruct (prefix/context, n = 116 per model-arm), and the two-readout form sits far closer to the free map (median operator cosine 0.518 vs 0.062, informative units) — consistent with a changing readout, not decisive. The fixed-eff-rank companion flips 7 of 504 verdicts. Standing baselines: free-map kNN acc@1 median 0.015 (chance 0.002); identity+bias median R² −0.72.

Fitted context-side transfer maps are near-full-rank corrections, not translations plus a low-rank term

Weakest sufficient class of the context-side map M (nested ladder collapsed to three families) by pair class on the 184 informative units, beside each unit's correction effective rank against its full-affine-over-translation held-out R² gain.

Figure. Full affine is weakest-sufficient for every framing-changing pair in both models; identity and provenance pairs sometimes need only a translation; no intermediate rank class ever wins. Filled = informative, open = validity-screened; dotted line = full dimension d = 3,584.

Framing-changing pairs always need the full affine map (98 of 98 framing, crossed, and user-framing informative units); translation suffices on 26 of 51 identity, 16 of 24 provenance, and 11 of 11 identity-vs-user units — a structural echo of the recovery-fraction headline, read under the null-degeneracy caveat above. No intermediate rank class ever wins: median effective rank of M − I is 3,489 of 3,584 on informative units, and rank k ≤ 128 corrections reconcile 0 of 222 rung-≥7 pair-arms context-side and 3 of 222 answer-side (k = 16-64, all instruct prefix). Correction directions overlap the diff-of-means direction modestly (median max cosine 0.33), and cross-pair correction subspaces beat the random-subspace null on 82-86% of combinations at small magnitude (medians 0.015/0.037, null 0.0096) — mostly pair-specific structure, no low-rank shortcut under this estimator.

Base-instruct same-condition maps reconcile only at full reparameterization, uniformly across the lattice

Left half of the figure: all 22 informative cross-model units (valid arm, positive target ceiling): rung-9 recovery fraction against the target ceiling, labeled by condition; filled = reconciles at rung 9. Right half: cross-condition correction-subspace overlaps against the random-subspace null.

Figure. Every informative base-instruct pair reads rung 9; correction subspaces overlap above the null but weakly. Circles prefix / squares context; teal = base to instruct, brown = instruct to base; dashed = the 0.90 bar (left) and the null p97.5 (right).

All 22 informative units read rung 9 in both directions (identical rung distributions each way); 8 of 22 reconcile at rung 9 (median rung-9 recovery over all 22 informative units 0.82, range 0.41-1.09), the rest reconcile at no rung. The derived-vs-free verdicts on valid arms are 59 of 60 transfer-map-insufficient (the battery's 12 shared-readout-supported verdicts all sit on construct-invalid user arms: 8 context, 4 plain-text prefix). The uniformity read is mixed: 99-100% of condition-pair correction-subspace overlaps beat the random null, but at small magnitude (medians 0.015/0.051 vs null 0.0096) — the base-instruct relationship is qualitatively uniform (always full reparameterization) without a strongly shared correction subspace. Scope: each side's answers are its own model's on-policy text, so this compares model-specific answer distributions, not text-matched answers.

The well-posed re-fit refutes the estimation-noise account: informative rung-1 units never read shared-readout-supported

Shared-readout-supported rate over the 256 parent rung-1 pair-arms, free-map-uninformative units excluded (denominators 216 ambient, 212 reduced), ambient fits beside the well-posed reduced-basis re-fit, pooled and per band of the per-unit basis rank k, against the 50% calibration line.

Figure. Pooled calibration stays far below the 50% line well-posed; the one band that crosses is entirely construct-invalid user arms. Rung-1 shared-readout-supported rate, ambient (blue) vs reduced (orange), pooled and per k-band; dashed = the 50% calibration line.

With every fold at n_train ≥ 2k, the calibration rate moves from 13 of 216 (6.0%) to 34 of 212 (16.0%) — far below the line, and every supported unit is non-informative. Only the k ≥ 512 band crosses (32 of 32; ambient 8 of 32), but all 32 are construct-invalid user-pair context arms (self-predictions). On the 18 informative class-0-excluded rung-1 units the rate is 0 of 18 in both bases (all transfer-map-insufficient; reduced g1 −0.50 to −0.05). Class-0 movement is small: 40 units stay free-map-uninformative and 4 ambient-supported units join them. This lands the falsification branch — the conjugation construction, not ambient conditioning, is the defect — read conditional on the captured subspace (0.27-0.64 held-out variance on the 18).

Reduced-basis verdict flips concentrate on construct-invalid user arms while the free map strengthens

Per-unit verdict transitions from the ambient battery to the well-posed re-fit over all 588 derived-vs-free units (504 within-model plus 84 cross-model).

Figure. The diagonal dominates, and every flip into shared-readout-supported sits outside the informative set. Ambient verdict (rows) against well-posed reduced verdict (columns), unit counts over 588 units; 381 transfer-map-insufficient units stay put.

All 54 flips into shared-readout-supported sit on construct-invalid user arms (53 from readout-changed: 51 context, 2 prefix). On the 184 informative within-model units verdicts move only between negative classes — transfer-map-insufficient 169 to 148, readout-changed 7 to 28, supported 0 in both bases — and 55 of 60 informative cross-model units read transfer-map-insufficient well-posed (54 stay, 1 joins from readout-changed, 1 drops to class-0). Well-posedness helps the free map more than the derived construction — informative median free-map R² rises from 0.012 to 0.066 while the derived-minus-free gap widens (median g1 −0.27 to −0.43). Structure reads soften in degree, not kind: the median correction effective-rank fraction falls 0.97 to 0.80 on informative units, intermediate classes become weakest-sufficient on 57 of 184 (full affine keeps 75 of 98 framing units), and rank ≤128 reconciles 5 context-side and 14 answer-side of 222 deep pair-arms (ambient 0 and 3).

Recaptured user-slot reads order second-turn predictability by completion provenance

Held-out R² of each within-cell map from the corrected clean read slot (last token strictly before the second user turn) onto that turn, per completion provenance; point estimates — per-fold spread is not persisted in the round mirror.

Figure. Second-turn predictability is ordered by completion provenance under both target conventions. Held-out R² for the second user turn — instruct model, chat template, clean read slot, layer 19: constant-string fallback 0.69 turn-end / 0.82 mean-over-turn; Haiku-simulated 0.24 / 0.37; on-policy Qwen 0.07 / 0.16. Point estimates.

The ordering holds in all 8 framing-by-model cells on the clean-slot-to-turn-end read: constant-string fallback 0.63-0.77, Haiku-simulated 0.23-0.34, on-policy Qwen 0.00-0.07. Target conventions move levels, not order: mean-over-turn lifts instruct plain-text on-policy from 0.02 to 0.29 (no committed figure; round-summary numbers). The "lmsys" arm predicts the parent's constant-string fallback turn, not real user text — prefix real, predicted turn one fixed sentence — so its top rank does not contradict the real-u2 result above (real 0.23 below Haiku 0.38). All six on-policy cells missed the 0.80 yield floor; the chat cells kept 2.5% (instruct, n=266 rows) and 4.2% (base, n=473) — yields per the parent yield stats, row counts per the interim writeup, neither in the round summary — a sample-size caveat on every on-policy read.

The corrected first-turn read sits at the floor, and the prior exchange carries the second-turn signal

Held-out R² from the clean read slot onto the first versus the second user turn per provenance, turn-end target, instruct chat cell shown.

Figure. A first user turn is unpredictable; the second is not. Clean read slot, turn-end target, instruct chat, layer 19: first turn −0.002 (the constant-string and Haiku columns share one first-turn read) and −0.075 (on-policy); second turn 0.69 / 0.24 / 0.07. Point estimates.

Every cell with a first-turn read sits at the floor — R² −0.002 to −0.075 across chat and plain-text cells, both models; the story render carries no first-turn read (no clean slot exists before a quoted first turn — see the round row under Design). The prior exchange carries the signal directly: ablating the first exchange from the parent's context-to-answer read costs 0.04-0.09 R² per cell, while regenerating the first assistant reply on-policy instead of the corpus reply moves the Haiku-turn read only about −0.01 (no committed figure; round-summary bridge numbers). One measurement trap: the parent's straddle slot, token-merged into the turn's first word, fakes first-turn predictability up to 0.16 turn-end and 0.32 mean-over-turn — token leakage the corrected clean slot removes.

Assistant-to-user transfer fails below answer-side correction, while the speaker label alone is nearly free

Held-out R² at each of the parent's nine transfer tiers — assistant-chat source map onto user-chat targets per provenance; parent-round ladder data, instruct model, prefix arm, symlog axis.

Figure. Only answer-side corrections approach the user-turn ceilings. Parent-round ladder, instruct, prefix arm, layer 19: direct transfer −3.2 to −4.2 across series; rotation 0.41-0.53 and answer reparameterization 0.60-0.68 on the constant-string and Haiku series (on-policy 0.12-0.18); dashed lines mark each target's own ceiling — no series reaches the 0.90 bar.

The recapture round's naive frozen-map read agrees with the tier-1 failure: assistant-to-user transfer R² −3.32 (base) and −3.67 (instruct), user-to-assistant −1.10 and −2.18 — each barely above its own shuffled-target null (−3.47, −3.79, −1.14, −2.27) — while nearest-neighbor retrieval of the held-out turn stays above chance (cosine top-10 accuracy 0.011-0.012 against 0.003 chance, assistant-to-user). The cost is the role, not the surface label: swapping the story speaker label between "Alex" and a literal "User" transfers at 0.70-0.74 (constant-string), 0.22-0.23 (Haiku), and ≈0 (on-policy) — within 0.04 of each cell's own read (no committed figure; round-summary numbers). Because the figure plots parent-round data, its constant-string and on-policy series inherit the parent defects: the parent's on-policy cells duplicated the Haiku arm's stored data, so this recapture round is the first genuine third arm.

---

Repro: No training. Fits: scripts/issue1689_fit_ladder.py (torch fp64, numpy-equivalence gated) via scripts/issue1689_ladder_parallel.sh on pod-1689 (4×H100, ~254 GPU-h including crash-fix rounds R9-R16 vs 46 budgeted). Analyzer digest + figures: scripts/issue1689_analyzer_digest.py, scripts/issue1689_analyzer_figures.py at branch commit 089b46a530cce72e7f81aa3e2dd4b757709dc76d (run commit 052a2c1cb32904b975a763235eec7e9ba6645a82); fig7 regenerated with a corrected title at 951942f545787d6101e2337879b696f8e0294784 (revision round 2). Eval JSONs: eval_results/issue_1689/ (ladder/, percell/, onpolicy_stats/, analyzer/pair_digest.csv). HF data repo (verified live via list_repo_tree, pinned to revision d1010a25f81ce184f68a9cc0ed49bce9736b80dd): activation store analysis_tensors (42 cells × 4 layers), raw completions raw_completions (kept rows only), rendered rows rendered. Reused artifacts: issue1310_common.PERSONAS verbatim (Wren/HELIOS/Dana; Vex excluded per plan §11) — fit: same character definitions as the sibling lattice line; LMSYS filter recipe from the #825 Track-M pipeline — fit: same corpus family and construct. The four pod-generated hero1_rung_heatmap_* figures at the run commit are superseded by fig1_rung_heatmap_* (they plot the raw rung code, which conflates no-reconcile with rung 9, and carry no validity overlays). Scope deviations vs plan: layer 19 only (14/18/26 captured, never fit); a single conversation-grouped 80/20 split per pair rather than per-rung 5-fold CV; bootstrap 1000 → 200 (R15) → 0 (v75 user descope; 76 pairs retain 200-draw CIs — 39 base, 37 instruct; the results-marker count of 79 overcounts by 3); nulls 200 → 40; the within-cell R² ≥ 0.5 pair-inclusion gate named in plan §6.6 item 4 was not enforced (all 126 pairs fit regardless of ceiling — this analysis screens post hoc instead); the plan-named mean-shift-vs-activation-spread read (plan §6.6 item 8) was not computed — it needs the HF activation store and stays a follow-up; the plan §5 secondary generation seeds 137/271 diagnostic-pair-subset stability check never ran (no artifact, no run event) — all generation-dependent cells use generation seed 42 only, so rung-headline stability under generation randomness is untested; the per-pair Procrustes operator cosine (addendum item 1b) landed in the 2026-07-29 zero-GPU follow-up round: scripts/issue1689_procrustes_battery.py at commit aeee660155, outputs at 9319318909 (eval_results/issue_1689/procrustes/), figure at 61eaa02e20. The wider-λ ceiling re-check (second follow-up round, 2026-07-29) ran scripts/issue1689_lambda_recheck.py on the shared VM (CPU, 0 GPU-h), outputs at commit 3c5053faf3 (eval_results/issue_1689/wider-lambda-ceilings/); figure by scripts/issue1689_lambda_recheck_figure.py at 284912d109, committed to main at 6d6f28e7a1. The derived-vs-free-answer-map round (third follow-up round, 2026-07-29/30) ran on the fellows lane (8×H200) in two jobs — 15194 (6h58m) and, after a checkpoint-key fix, the full-coverage re-sweep 15440 (4h55m) — realized ≈ 95.1 GPU-h vs the 81 re-gated. Drivers at branch commits c9bcbb254d / e9e398aabf / 7cd79eb379 plus staging hot-fix e12726920d; initial battery outputs at 3291f11d51; digest + fold-figure scripts at 40da4290aa; model-qualified key fix at d63ec44ac7; re-sweep outputs (504 of 504 in both within-model batteries, 0 failed, 0 key-mismatched; Gate-1 parity re-PASS 0.000e+00) at 956457e561; full-coverage digest + figure update at 86096e99af; fold figures committed to main at 276a44dace. Parity gate PASS (max abs rung-R² diff 0.000e+00 against the published parent pair-arm); factor bundles verified on the HF data repo at derived_vs_free/analysis_tensors (1,681 files: 504+504+84+84 current unit bundles, 252+252 superseded pre-fix names, one probe). The well-posed re-fit round (fourth follow-up round, 2026-07-30, plan v10) ran on the fellows lane (8×H200) as job 15738 (completed in 51m27s, ≈6.9 GPU-h of 12 budgeted) after one crash-fix relaunch — job 15724's fence step needed parent per-unit inputs the SLURM rsync lane never ships, so fix 269792a171 stages them from the HF mirror prefix issue1689_speaker_lattice/parent_inputs (1,180 files, staged in-run at data-repo revision bdb45ccbbe87); round drivers at 45c250942f plus the staging fix; battery outputs, the paired digest, and the 19 round figures committed at e006ba1b9484; Gate-1 ambient no-op parity max abs scalar diff 0.0; reduced-basis unit bundles verified live on the HF data repo under the four _wellposed subtrees of derived_vs_free/analysis_tensors (504 + 504 + 84 + 84 files, exact-set checks in-run and re-listed at write time). The real-u2 capture round (fifth follow-up round, 2026-08-03/04, real-u2-capture label) ran on the fellows lane (SLURM job 18578, completed 2026-08-04T00:15:46Z): ~3,800 real multi-turn LMSYS+WildChat conversations sampled via the #1738 eligibility machinery, rendered into user-identity cells × 3 framings × 2 models × 2 provenances (real u2 + haiku-simulated u2 on the same 3,800), captured teacher-forced at L19 under the user_slot_recapture conventions, then fit with within-cell ridge + identity+bias baseline + kNN retrieval + 40-draw shuffled null. Fits SHA 376200beb71839c7583168cda0223629b71fd565; outputs at eval_results/issue_1689/real_u2_capture/ (per_cell_summary.json, rung_reached_matrix.json). Round figures fig14_real_vs_haiku_prefix_r2 (hero) and fig15_real_u2_r2_lattice (per-cell lattice) generated by scripts/issue1689_real_u2_figures.py and committed at e1d6e27d003c81ba27db36fb9bb967a929480ad8. Companion doc update: docs/results_summaries/2026-08-02-framing-character-user-turn-map-transfer-filled.md Result 3 should be revised to note the real-u2 finding — deferred to that doc's next revision. Per-pair CI claims are restricted to the draw-bearing subset; the 0.85/0.90/0.95 bar sweep uses point estimates for all pairs. Identity space is 3 characters + assistant + user (one character per category), an external-validity limit on every identity claim. The user-slot recapture round (run 2026-07-30/31, label user_slot_recapture; body fold deferred to 2026-08-09, landed via task #2031) ran on the fellows lane as SLURM job 16221 (completed 2026-07-31): re-captured the parent's user-identity cells at corrected token boundaries (clean read slot, last token strictly before the turn) plus bridge cells, with the parent's three realized defects fixed; readout at eval_results/issue_1689/user_slot_recapture/summary.json (committed 948fa8c6be from the HF mirror issue1689_speaker_lattice/user_slot_recapture/eval_mirror/); fig16-18 reuse-copied at a198d81ef8f3db382333c83806bf768ac2a76afe from the 2026-08-02 results-summary render (scripts/results_summary_framing_plots.py at abd729ecc9).

Context: Task #1689, a follow-up to #825 (the Track-M LMSYS context-to-answer map line whose corpus filter this task reuses), created 2026-07-25 from the verbatim user prompt: "i want a controlled comparison between: assistant with chat template / assistant without chat template / assistant in story / user in chat template / user without chat template / user in story -- for user it should be either: real user generated turns from LMSYS, always haiku generated, always on-policy generated by Qwen itself / other AI character with chat template / other AI character without chat template / other AI character in story / other assistant-like human character with chat template / other assistant-like human character without chat template / other assistant-like human character in story / other random character with chat template / other random character without chat template / other random character in story. (all except user should always be on policy generated) all in both instruct and base model. all characterized according to the tiers of mapping similarity. [then] run the full thing in the background with happy coder". Run 2026-07-26 through 2026-07-28 (crash-fix rounds R9-R16; torch/GPU fit port in the user session). The framing-by-identity interaction addendum (the crossed-vs-marginal and side-localization results) was user-approved scope posted on the task 2026-07-28, before analysis. A zero-GPU follow-up round (2026-07-29) folded in the addendum's per-pair Procrustes operator-cosine battery (orchestrator-run; no separate user prompt). A second zero-GPU follow-up round (2026-07-29, label wider-lambda-ceilings, proposer-initiated cheap band, no separate user prompt) re-checked the ridge-λ grid ceilings; scope verbatim: "Wider ridge-lambda ceiling re-check — extend the ridge grid from logspace(-2,4,13) to logspace(-2,7,19) and refit the 84 per-cell L19 within-cell maps (both models x 21 conditions x 2 arms); recompute ceilings + the reach-bar-dependent rung reads for the informative pairs." A third follow-up round (2026-07-29, label derived-vs-free-answer-map, source user-chat) tested the mentor-posed shared-readout question — one readout plus one affine context-space transfer map, with the derived answer-space map compared against the free map — plus the context-map structural battery and the 21 base-instruct same-condition cross-model pairs; the verbatim origin prompts are recorded in the task's followup-scope v3 marker note. A fourth follow-up round (2026-07-30, label wellposed-shared-readout, proposer-initiated cheap band, no separate user prompt) re-ran the three fit batteries in a per-unit well-posed reduced basis to disambiguate the derived map's calibration failure between ambient estimation noise and a construction defect; the verbatim proposal spec is recorded in the task's wellposed-shared-readout followup-scope marker note. A fifth follow-up round (2026-08-03/04, label real-u2-capture, source user-chat) supplied the exact-comparability real second-user-turn arm the parent's lmsys cell had held only as a constant-string fallback; verbatim origin prompt (2026-08-03 chat on the framing/character/user-turn report, following "do we not have actual real user data?"): "do a and b" — where the arm delivered here filters LMSYS/WildChat to conversations with ≥2 real user turns, renders into this issue's rig, captures the real u2 at the recapture round's slots, and fits the user-turn maps. Corpus: ~3,800 real multi-turn conversations. Render: user-identity cells only, 3 framings × 2 models, user_slot_recapture render conventions. Companion arm: haiku-simulated u2 on the SAME conversations. Capture: teacher-forced L19, X_prefix clean slot + turn-end + mean-over-turn targets. Fits: within-cell ridge (reduced-basis rule where n_train < 2k applies), identity+bias baseline, kNN retrieval, shuffled-target null, provenance-transfer reads real↔haiku, first-vs-second-turn control on the real arm. Fold directive: correct the promoted body's tier-1-real-data claim (deferred from the recapture round and the 2026-08-03 inline rounds, incl. the real_u2_1738_reuse read) and update Result 3 of docs/results_summaries/2026-08-02-framing-character-user-turn-map-transfer-filled.md. A sixth recorded follow-up round (run 2026-07-30/31 — chronologically before the real-u2 round — label user_slot_recapture, inline user-override round) re-captured the user-slot cells with the parent's realized defects fixed; its readout fold into this body was deferred from 2026-07-31 and landed 2026-08-09 via task #2031.