The DPO→RLVR step of the Llama Tülu ladder barely moves the linear context→answer map: at most a rotation on seven of eight corpora, and the smallest pooled R² step on the ladder (MODERATE confidence)

Methodology: docs/methodology/issue_1336.md · gist mirror

Takeaways

• SFT installs the linear context→answer map; RLVR barely moves it. Pooled on-policy held-out R² at layer 30 rises .413 → .578 → .598 → .602 → .619 across base/SFT/DPO/RLVR/longer-RLVR — the DPO→RLVR increment (+.004) is the smallest step on the ladder against +.165 at base→SFT — and per corpus (round 3) DPO→RLVR closes at tier 0–5 on 7/8 corpora (6/7 counting the two Tülu-derived corpora as one); the only corpus no tier closes (IF-constraints) and the only one needing the rotation (MATH) are exactly the two RLVR trained on.
• The base map never reconstructs a post-trained map on the models' own text (tier-8 gaps +0.14 to +0.26 per corpus in round 3; +0.12 to +0.28 pooled, IF-constraints outlier +0.45) — but on fixed target text the pooled gaps shrink to +0.03–0.14 (IF-constraints base→longer outlier +0.22), so much of the on-policy irrecoverability rides on the text distribution each checkpoint writes, not only on map coordinates.
• No cluster-level teaching signal survives: at every adjacent transition the most-improved prompt cluster on own text is indistinguishable from the selection-matched permutation null given the variance (p = .14/.08/.39/.17; null bands sit well below achievable ceilings, so the test is informative), and the off-policy arm's rejections never agree with the on-policy arm on which clusters moved.
• Longer-RLVR (Tülu-3.1) accumulates more coordinate change than the single RLVR step, not less — DPO→longer-RLVR moves 4/8 corpora to a higher sufficient tier in round 3, and every pooled post-training transition into longer-RLVR leaves IF-constraints tier-8 gaps of +.20 to +.26. The map moves with RLVR dose.
• The fitted map wins held-out R² everywhere (identity+bias baseline −1.2 to −3.5) yet identity+bias wins retrieval at rank 1 in nine of ten pooled cells (.604/.615 vs ridge .564/.602 on the RLVR on-policy cell) — the documented R²-vs-retrieval dissociation, so every map-strength claim here is R²-specific and does not assert the fitted map is the better retrieval reader.
• Validity bounds. The pooled round's fold-reproducibility gate formally failed its band (Δ_assign +0.077 vs the 0.052 tolerance) and the run proceeded on an instrument-conservatism adjudication, so pooled values inherit ~.05–.07 of conservatism and are never numerically compared to round-3 per-corpus fits; one released ladder, single fold/corpus seeds, within-run bootstrap CIs only; the triggered leakage-excluded sensitivity read is unrun.

Goal

• This experiment in context: #825 found on Qwen2.5-7B that post-training does not create the linear context→answer map: the pretrained model carries it at 87% of instruct strength, and the instruct map is reconstructible from the base map by a general linear change of coordinates to within 0.0003 R² (map construct from #779). Qwen2.5 predates RLVR, so #825 tested only SFT/DPO-era post-training. Round 1 extended the read to the only released RLVR-bearing ladder (Llama-3.1-8B Tülu-3) at 5,000 prompts. Round 3 scales the corpus scope 3.8× on GSM8K, 3× on LMSYS, and adds four stage-matched corpora (MATH, IF-constraints, UltraFeedback, Tülu-3 SFT-mix), and replaces the single reparameterization gap with a 9-tier metric ladder (direct transfer → offsets → scaling → rotation → linear reparameterization of contexts / answers / both) — so "elicitation" resolves into which correction class suffices per stage-pair × corpus. Round 4 (pooled-multidataset-onoff-policy-stage-transfer) pools the seven chat-render corpora into one group-aware split and adds an off-policy training-text arm — testing whether the stage ordering survives a single pooled fit, and whether each checkpoint's map is tied to the text it writes itself. Round 3 diverges from #825 in that the base map here does NOT reconstruct any post-trained map at t ≤ 8 (Δ_tier8 +0.14 to +0.26); a cross-model divergence (Llama-3.1 vs Qwen-2.5), not a contradiction of the elicitation reading, and a second RLVR checkpoint tests dose.
• Broader narrative: This serves the contextual-vs-base identity question (docs/open_questions.md, identity-contextual-vs-base anchor): does post-training teach new context→answer structure, or re-expose structure the pretrained model already has? RLVR on verifiable rewards is the strongest published candidate for genuine teaching, so a tight null across the DPO→RLVR pair — plus the tier-ladder resolution that SFT→DPO needs a strictly higher tier than DPO→RLVR on 5/8 corpora, and that the two DPO→RLVR non-null corpora are exactly the RLVR training-mix distributions — is the most informative single read for the elicitation-not-teaching account. Two caveats bound it. First, DPO→longer-RLVR shows the map DOES move with further RLVR training on 4/8 corpora, so "RLVR doesn't change the map" is really "the single Tülu-3 RLVR step doesn't move the map beyond a rotation on the corpora it wasn't trained on". Second, a single ladder cannot separate "RLVR doesn't teach new coordinates" from "RLVR training on this ladder used a smaller effective weight update than the SFT step" — plausible because RLVR literature suggests low-rank effective updates; separating them would need a second ladder or a paired weight-space read.

Methodology

Design: Five released checkpoints of one ladder — meta-llama/Llama-3.1-8B (base), allenai/Llama-3.1-Tulu-3-8B-SFT, allenai/Llama-3.1-Tulu-3-8B-DPO, allenai/Llama-3.1-Tulu-3-8B (post-RLVR), allenai/Llama-3.1-Tulu-3.1-8B (longer-RLVR) — crossed in round 3 with eight prompt corpora and one render format per corpus, giving 40 generation cells and 56 stage-pair × corpus × layer-30 metric-ladder pair files (7 stage-pairs × 8 corpora). The post-training stage is the single manipulated variable; corpora, renders, sampling, and the fit recipe are held fixed. Round 3 corpora (revision-pinned): chat render on GSM8K test (1,319), GSM8K train full (7,473), MATH (7,500 from the RLVR mix), IF-constraints (11,000 sampled from the RLVR mix's ifeval component), UltraFeedback prompts (11,000 from the Tülu-3 8B preference mixture), Tülu-3 SFT-mix (11,000, stratified 3,667/3,667/3,666 across wildchat/flan/evol-codealpaca; corpus slug sft11k), and LMSYS-Chat-1M (23,000 sampled to yield ~15,000 kept); naturalistic render on LMSYS only. Rounds 1–2 (5,000 LMSYS × 2 renders + 5,000 GSM8K train + 1,319 GSM8K test) are superseded by the fatter round-3 corpora except the GSM8K test cell, which is retained as the decontaminated companion under the n<d caveat below. Per the standing both-mappings rule, the context-based mapping ran on every corpus × format; the prefix-based mapping ran on the naturalistic LMSYS render only — the chat-template prefix slot is row-constant by construction (measured max pairwise cosine distance up to 1.5e-4), so its map is degenerate at every stage. The naturalistic prefix mapping ran and reads R² 0.001–0.005 across all five stages (base 0.005; SFT/DPO/RLVR/longer-RLVR all 0.001) — floor-limited, invariant across stages, carried as an uncovered signal cell (a scope caveat, not a null).

Round 4 (follow-up pooled-multidataset-onoff-policy-stage-transfer) re-fits the same five checkpoints on one pooled corpus: the seven chat-render corpora pooled into a single group-aware 80/20 split (the naturalistic LMSYS render is excluded — the same prompts rendered twice must not be pooled). Two training-text arms per checkpoint: on-policy (the checkpoint's own sampled answers — the round-3 generations reused verbatim) and off-policy (the answers the four other checkpoints wrote to the same prompts, teacher-force-captured through the fitting checkpoint and concatenated; 20 new off-diagonal capture cells). This gives 10 pooled cells (5 checkpoints × 2 arms), plus matched-n companions at 15,000 train rows to separate sample-size effects from stage effects, a pooled transfer battery (own fit / direct transfer / context-side / both-sides reparameterization per transition × corpus), and a per-cluster battery over the split groups. The plan's generic preference-tuning slot is realized as DPO — no open checkpoint family ships base/SFT/PPO-style-preference-RL/RLVR — an availability constraint carried as a scope caveat.

Training: N/A — no model training. Round 3 spent 194.37 GPU-h against a budgeted 98 GPU-h (2.0× over budget; deviation from plan §11, recorded here; the fit + metric-ladder phase's per-cell wall was under-estimated at plan time — the pilot batteries measured 541s/corpus vs the planned 1.4 h/corpus, but the mass of stage-corpus cells at 15–17k rows in generation + capture drove the wall). Round 4's run digest records 2,126.15 GPU-h realized against 210 budgeted (10.1× over; recorded here as a plan deviation) — the figure aggregates every round-4 attempt on the fellows H200 lane, including an off-policy capture job that ran far over its planned per-cell basis before a resume-check bug halted it, and two fit jobs lost to memory failures and relaunched.

Evaluation: Per (stage-pair p, corpus s, layer L=30) the reparameterization ladder produces held-out R² of the source map applied to target contexts at every tier t ∈ {0..8}, on the same rows and folds as the target's within-stage map. Tiers, per docs/mapping_similarity_metrics.md: t0 direct transfer (ŷ = W_s x + b_s applied unchanged), t1 context offset (ŷ = W_s (x − Δx) + b_s, Δx from cloud means only), t2 answer offset (Δy on the output side), t3 bias offset (refit b* only), t4 global scaling (α W_s x + b*), t5 mapping rotation (R W_s x + b*, orthogonal Procrustes on directions), t6 linear reparameterization of contexts, t7 linear reparameterization of answers, t8 both (A ∘ W_s ∘ B). All offsets fit on clouds alone (never on paired context→answer rows). Δ_t = within-stage R² − tier-t R². The sufficient-tier statistic is min{t ∈ 0..8 : Δ_t ≤ 0.021 · ex_v2 = 0.0207} at the paired prompt-level bootstrap point estimate; "none ≤ 8" is a reportable value. Primary scale is raw pooled R² (round 3's fatter fold-train regime makes the recalibration rescue used in rounds 1–2 unnecessary — G0v2/G1v2 both PASS at raw); the held-out cross-fitted per-dim affine recalibration companion is always computed. The recalibration companion adds 4,096 per-dim scalar shift+scale parameters cross-fitted from folds, so a genuine map-mismatch signal can be partially absorbed by it; recal disagrees with raw on 3/8 corpora for DPO→RLVR (GSM8K test raw t2 → recal t0; GSM8K train raw t3 → recal t0; IF-constraints raw "none" → recal t8), so raw is the binding read and recal is reported as the companion that a per-dim rescaling accommodates. The stage-of-record contrast C_v2 = Δ_tier8(dpo→rlvr, gsm8k_train_full, chat) on the RLVR training distribution: point −0.0031 with a bootstrap 95% CI running from −0.0039 to −0.0024 on the raw scale (a signed negative CI — reparameterized-DPO reconstructs RLVR marginally better than the within-RLVR fit on that corpus; magnitude 5× smaller than the 0.021 elicit band, sign is direction-of-effect, not "matches to zero"). Decision constants (round-3 revisions with ex_v2 = S_qwen_v2 / 0.6731 = 1.0303, S_qwen_v2 = 0.6935): elicit_band_v2 = 0.0207; practical_scale_v2 = 0.0518; health_gate_v2 = 0.0515. G0v2 (Qwen anchor + Gram-vs-primal parity) PASS at abs_dev 5.9e-6 and 2.2e-16 primal-vs-Gram agreement. G1v2 (RLVR × LMSYS23k chat vs 0.20606 bar) PASS at raw R² 0.6201 / recal 0.6200. Capture-parity gate (100 rows × 32 layers) PASS at min mean cosine 1.0000005 vs 0.999 threshold. All three gates were set in plan v13; no descope fired. No LLM judge is involved: the DV is an activation-geometry read.

Round-4 evaluation (pooled split): per checkpoint × training-text arm, one pooled ridge map is fit across the seven chat-render corpora on a group-aware 80/20 split — 173 groups (per-corpus MPNet sentence-embedding sub-clusters, merged across corpora only by near-duplicate union-find), whole groups assigned to one side, per-corpus test share 0.21–0.26 against a 0.15 floor. Headline numbers are held-out R² on the 20% test partition at the frozen layer 30; a train-side 5-fold CV companion with bootstrap CIs is tabled below — the two conventions differ by up to .024 across the ten pooled cells, and the largest divergence falls on the headline cell (longer-RLVR on-policy .619 test-partition vs .596 CV; base .413 vs .394) — and are never mixed in one comparison. λ is selected by inner-group CV on the 23-point grid spanning 1e-3 to 1e8 with zero edge extensions; realized selections at layer 30: 3,162 on-policy (10,000 for the base cell), 10,000 off-policy. n_train/d = 37,491/4,096 on-policy and 149,964/4,096 off-policy (both n > d; the matched-n companions at 15,000 are also > d); n_test = 10,612 and 42,448. The per-cluster battery computes delta-Q — the mean, over a cluster's held-out prompts, of the per-prompt change in R² from the source to the target checkpoint under the pooled context-side-reparameterization read — with a max-cluster statistic tested against a within-corpus block-permutation null (1,000 draws, the max-cluster selection applied inside each draw), the null band and the achievable per-cluster ceiling both drawn in the figure. The stage ordering is stated at layer 30 throughout; the max-over-layers ordering differs (the SFT cell peaks at .611 vs RLVR's .590), so the pooled stage story is layer-convention-dependent and layer 30 is the frozen headline convention.

Round-4 health gates — the fold-reproducibility gate failed and the run proceeded on an adjudication. G0v3 (fold-membership reproducibility: fits on identical pooled LMSYS train rows, the group-aware fold assignment vs three size-matched seeded random assignments, each fit read at its own argmax layer over the frozen set) returned Δ_assign = +0.0766 against the 0.0515 tolerance — verdict FAIL-leakage-exceeds-band, a formally failed gate. The run halted, ran the plan-named discrimination at the halt, and proceeded on an autonomous instrument-conservatism adjudication (persisted in gates_v3/g0v3_adjudication.json; task marker v276): grouped R² .5383 vs matched-random mean .6149 (across-draw SD .0002); the discrimination attributed 0.05–0.07 of the excess to family-holdout conservatism, bounded any leakage-removal component near .01 (ceiling .032), and found no defect signature — all four fits argmax at layer 30, fold sizes matched the manifest profile exactly (seed 13360), and the retained report-only level diagnostic (grouped .5383 vs the round-3-style reference .6090, difference .071) is construction-confounded. The halt is narrated as a grouping-effect excess (family holdout legitimately hardens generalization), never as established leakage. Two consequences bind every pooled number in this body: the gate certificate is corpus-local — it ran on the LMSYS slice (10 test-side groups), not on MATH (5 test-side groups) or IF-constraints (4), where the two confirmatory per-corpus gaps live; and the pooled headline inherits roughly 0.05–0.07 of instrument conservatism relative to round-3-style row-level folds, so pooled and round-3 R² values are not directly comparable and are never quoted side by side without this protocol delta. The other two gates passed: the pooled rig-health kill gate at raw best .590 against the .206 bar, and the off-policy capture-parity gate (100-row recapture of one off-diagonal cell) at min mean cosine .9999991 against .999.

Round-4 coverage: all 10 planned pooled cells and 20 transition pair files completed. The first fit job lost the five off-policy units to two memory failure modes (a host cgroup out-of-memory kill and a CUDA out-of-memory in the λ sweep); a relaunch with both modes fixed recomputed all five (job completed exit 0:0) and reused the four finished on-policy units via done-markers — resumed bootstrap chunks were verified equal to the pre-crash arrays (numpy array-equal check), so cross-unit comparability is preserved. One declared discard: the capture-parity recapture tensors (gates_v2/g2_recapture/*.pt), regenerable from the persisted rollout text (marker v310). Two pre-relaunch review concerns were closed by the realized run and are recorded in the concern ledger: the committed adjudication record was staged to the launch tree and the gate step consumed it (the run advanced past the halt into production compute rather than re-halting), and the queue-width cap the review mandated was armed (width 2 for the off-policy fit units) with zero out-of-memory events across the recovered run.

Round-4 sensitivity reads carried as limitations. The leakage-excluded sensitivity R²'s trigger fired (104 test rows carry a ≥0.90-cosine cross-corpus train twin; 3.2% of both Tulu-derived slices) but the read is not computable from the persisted prediction matrices — they carry predictions only, no ground-truth answer vectors — and re-running needs tens of GB of activation staging; it is carried as a named follow-up. The cluster-resampled sensitivity CI on the MATH / IF-constraints gaps is unrun for the same staging reason, and with 5 and 4 test-side groups on those corpora a group-level bootstrap would itself be unstable. The answer-text divergence diagnostic behind the off-policy arm (per-checkpoint exact-match rate and similarity of generated answers) is unrun, so off-policy reads are conditional on unmeasured cross-checkpoint text divergence — strongest on MATH/GSM8K, where correct answers converge across checkpoints. The planned regression-to-the-mean check on top delta-Q clusters is moot — no cluster survives the null. The plan's pooled-reproduces-per-corpus slice comparison (each pooled per-corpus slice within round 3's bootstrap CI on at least 7 of 8 slices) is not adjudicated: the failed gate established a 0.05–0.07 systematic depression from family holdout, so pooled slices sit below round-3 values by construction; and the UltraFeedback and SFT-mix slices count as one quasi-unit in any such slice tally (see Data extraction), which would cap the effective count regardless.

Complete constants (round 3 additions in bold; unchanged from plan v13 unless noted):

Parameter	Value	Source
Ladder checkpoints	meta-llama/Llama-3.1-8B; allenai/Llama-3.1-Tulu-3-8B-SFT; allenai/Llama-3.1-Tulu-3-8B-DPO; allenai/Llama-3.1-Tulu-3-8B (post-RLVR); allenai/Llama-3.1-Tulu-3.1-8B (longer-RLVR)	arXiv 2411.15124 + Hub card-metadata lineage (plan v13 §11)
Round-3 eval rows (kept, per stage)	~15k LMSYS chat + ~15k LMSYS naturalistic (from 23k sampled); 7,473 GSM8K train (all); 1,319 GSM8K test; ~7.4k MATH; ~9k IF-constraints; ~9.5k UltraFeedback; ~9k Tülu-3 SFT-mix	plan v13 §4 data table
Sampling	T=1.0, top_p=0.95, max_tokens=1024, seed 42, 1 sample/prompt; vLLM max_model_len=4096	parent generation script SamplingParams (plan v13 §11)
Render	Tülu chat template applied as text to all 5 checkpoints (token-id-verified); naturalistic = template stripped	plan v13 §11
Ridge fit	primal (d-space) ridge at n_train > d; Gram-space at n_train ≤ d (byte-preserved); K=5 folds, fold seed 0; pooled R² with fold-local test means	round-3 core fix, plan v13 §4 §5
λ grid (round 3)	np.logspace(-3, 8, 23) + adaptive edge rule (≤2 decade-extensions per side, then estimator-limited label)	project LAMBDAS_N1M constant; plan v13 §4
λ selection (round 3)	inner-group-CV, N_INNER_LAMBDA_FOLDS=2; pure-GCV at n_train < d refused by shared fit cores (_refuse_unguarded_gcv, GCV_DOF_CAP=0.9 default)	project registered GCV-guard selector; plan v13 §4
Row filters	non-empty first user turn; Tülu prompt-token budget ≤ 3,072	PROMPT_TOKEN_BUDGET parity from the parent capture rig
Metric ladder	9-tier battery: direct transfer → context/answer/bias offsets → global scaling → orthogonal Procrustes rotation (direction-aware) → linear reparam of contexts / answers / both	docs/mapping_similarity_metrics.md; plan v13 §3
Activation dtype	bf16 capture + turnstore; held-out prediction matrices fp16	parent (plan v13 §11)
Layers	full 32-layer sweep; frozen report set {16, 21, 22, 30}; headline layer 30	fractional-depth remap of the parent frozen set (plan v13 §11)
Bootstrap	1,000 paired prompt-level draws, shared across tiers within a pair (seed 5000 + eval-set index)	parent convention (plan v13 §11)
Shuffle nulls	20 per fit	parent convention (plan v13 §11)
Fold seed / corpus sample seed	0 / 1336 (single seed at each — CIs are within-run bootstraps, not between-seed replication)	plan v13 §11
Round-3 bars	s_qwen_v2 = 0.6935; ex_v2 = 1.0303; bar_v2 = 0.2061; elicit_band_v2 = 0.0207; practical_scale_v2 = 0.0518; health_gate_v2 = 0.0515	G0v2/v2_bars.json; plan v13 §3
Identity+learned-bias baseline + kNN retrieval	reported per fitted map (dims match at 4096→4096; per-cell values in the dissociation result and the pair-file baselines blocks)	project mapping-baselines standing rule; plan v13 §4 §5
vLLM generation cap-hit fraction (finish_reason == "length") per stage/corpus	NOT aggregated in round 3 — the 40 answers.jsonl files carry finish_reason per row but the per-stage cap-hit rate was not tallied at generation time; recorded as a CLAUDE.md standing-rule gap and staged as a cost_class: free-analysis follow-up. Seed-42 5-row spot check on the GSM8K train RLVR file found 0/5 finish_reason == "length".	CLAUDE.md § After Every Experiment item 8; body free-analysis follow-up list
Round-3 compute	194.37 GPU-h (vs 98 budgeted; 2.0× deviation) on Charmander H200 (fellows lane), 8-GPU sharded, ≈24 h wall	run digest epm:results v7, plan v13 §9

Round-4 (pooled split) constants:

Parameter (round 4)	Value	Source
Pooled split	173 groups (per-corpus MPNet sub-clusters + near-duplicate union-find merges); whole-group assignment; per-corpus test share .207–.255 (floor .15); 48,103 pooled rows post-dedup	analysis_tensors/pooled_split_v3/split_manifest.json
Near-duplicate quarantine	cross-corpus cosine ≥ 0.95; 239 edges → 306 rows train-locked (0.64% global; worst corpus 2.08%)	split manifest near_dup_audit block
n_train / d; n_test	on-policy 37,491 / 4,096 (test 10,612); off-policy 149,964 / 4,096 (test 42,448); matched-n companions 15,000	eval_results/issue_1336/cells_pooled_v3/*.json
λ selection (round 4)	inner-group CV, 23-point grid np.logspace(-3, 8, 23), zero edge extensions; selected at layer 30: on-policy 3,162 (base cell 10,000), off-policy 10,000	cells_pooled_v3/*.json selected-λ fields
Fold-reproducibility gate	Δ_assign = mean-of-3 size-matched random-fold R² − group-aware R², each fit at its own argmax over {16, 21, 22, 30}; pass band −0.01 to +0.0515 (0.05·ex_v2); realized +0.0766 → fail, adjudicated proceed	gates_v3/g0v3.json, gates_v3/g0v3_adjudication.json; plan §6 (v26)
Cluster delta-Q null	max-cluster statistic; within-corpus block permutation, 1,000 draws, selection inside each draw; paired-bootstrap seed base 5300	decision_v3/cluster_delta_q_per_transition.json; epm:results v311 card
Pooled CV companion (never mixed with the test-partition headline)	on-policy CV .394 / .574 / .594 / .592 / .596, 95% CI half-widths ≤ .003 (base [.391, .396], SFT [.572, .576], DPO [.591, .596], RLVR [.589, .594], longer-RLVR [.593, .598]); off-policy CV .550 / .458 / .463 / .464 / .463	cells_pooled_v3/*.json r2_bootstrap_ci_frozen_layers
Round-4 raw frozen-layer means	.494 / .515 / .511 / .546 (on-policy cells, fold-observed CV convention) over layers {16, 21, 22, 30}; layer 30 is the frozen headline layer	decision_v3 headline block
Round-4 compute	2,126.15 GPU-h recorded vs 210 budgeted (10.1×; aggregates all round-4 attempts — see Training)	epm:results v311 reproducibility card

Data extraction: LMSYS is lmsys/lmsys-chat-1m at revision 200748d9d3cd, rows 0–4,999 identical to the parent's track_s.jsonl (byte-asserted), rows 5,000+ continuing the same first-user-turn stream, exact-sha256-deduped, until 18,000 new unique prompts. GSM8K is openai/gsm8k @ 740312add88f, config main. MATH and IF-constraints are the two non-gsm8k components of allenai/RLVR-GSM-MATH-IF-Mixed-Constraints @ 7dbd180f5440 (7,500 MATH rows verbatim; 11,000 sampled from the 14,973 ifeval component, seed 1336). UltraFeedback prompts are the prompt field of allenai/llama-3.1-tulu-3-8b-preference-mixture @ 78a6f0078594, seed-1336 sample of 11,000 after exact-sha256 dedup. Tülu-3 SFT-mix is the union of three sources from allenai/tulu-3-sft-mixture @ b14afda60f1b, stratified 3,667/3,667/3,666 (wildchat/flan/evol-codealpaca), first user turn only. Every corpus was decontamination-screened against the 1,319 GSM8K test questions (normalized-exact match); the allenai mixes are decontaminated upstream per their model cards and the in-house verification found no matches. Answers were generated on-policy per model with vLLM (single stochastic sample per prompt at the sampling constants above); activations were captured teacher-forced on each model's own sampled text into bf16 turnstore shards. n/d hygiene at K=5: LMSYS fold-train ≈ 12k, IF/UF/SFT ≈ 7.2–7.7k, GSM8K-train / MATH ≈ 5.9k (all ≥ d = 4,096 except GSM8K test, which is retained only as the labeled n < d companion — its within-stage R² and tier reads run inner-group-CV with a 0.9 dof cap under the project's GCV-guard fit-core convention and are companion values only, never lattice-bearing). Realized keep rates: LMSYS 0.63–0.73; GSM8K/MATH 0.98–1.00; IF/UF/SFT 0.80–0.92 — all above the 0.80 keep-rate floor except LMSYS, carried as a coverage caveat. The plan-named companion figures keep_rate.png and dedup_sensitivity.png (committed under figures/issue_1336/) plot the per-corpus keep rates and the dedup threshold sweep.

Round-4 pooling: the seven chat-render corpora (71,906 built rows) reduce to a 48,103-row pooled panel — the five-way intersection of rows kept at every checkpoint, exact-sha-deduped across corpora (the dedup dropped 150 UltraFeedback/SFT-mix cross-corpus duplicates). Test rows per corpus: LMSYS 2,905 / GSM8K train 1,523 / GSM8K test 288 / MATH 1,689 / IF-constraints 1,479 / UltraFeedback 1,362 / SFT-mix 1,366; GSM8K test, at 288 rows in 2 test-side groups, is the thinnest slice and carries width caveats. Near-duplicate audit over the panel (MPNet cosine): 239 cross-corpus edges at the 0.95 threshold quarantine 306 rows to the train side (0.64%; per-corpus ≤ 2.08%), so gap magnitudes are defined over the corpus-distinctive test distribution; at 0.90 the edge list grows to 1,239 (783 incident rows), a threshold-sensitivity flag. Edge provenance: SFT-mix↔UltraFeedback dominates both lists (170 of 239 = 71% at 0.95; 744 of 1,239 = 60% at 0.90) — a shared-upstream Tulu-3 duplication signature, so those two slices are treated as one quasi-unit in per-slice tallies; the anticipated MATH↔GSM8K similarity channel is refuted (zero direct edges at either threshold); and GSM8K train↔test edges are zero, with GSM8K-test quarantine at 3 rows (0.23%), so quarantine censoring cannot materially manufacture a train-vs-test contrast — that caveat is retired as immaterial. The report-only cluster diagnostic (transitive closure over sub-clusters: 70 merged groups in 5 components, the largest spanning 38.5% of the pool; merged-mass fractions MATH .785, GSM8K train .709) documents the topical-entanglement backdrop for those two slices; only the 306 individually quarantined rows are actually excluded from the test side.

Sample training/evaluation data + completions:

1 of 5,000 rows (row 1309 — seed-42 spot-check row; random sample) from the After-RLVR model's GSM8K-train generations. Full artifact: answers.jsonl @ 8c54f9fc (round-1 wave; the round-3 fatter GSM8K train uses the identical openai/gsm8k questions extended to all 7,473):

{
 "prompt_idx": 1309,
 "prompt": "Martin has 18 goldfish. Each week 5 goldfish die. Martin purchases 3 new goldfish every week. How many goldfish will Martin have in 7 weeks?",
 "response": "To determine the number of goldfish Martin will have after 7 weeks, let's break down the problem step-by-step.\n\n**Initial Number of Goldfish:**\nMartin starts with 18 goldfish.\n\n**Weekly Changes:**\n- Each week, 5 goldfish die.\n- Each week, Martin purchases 3 new goldfish.\n\n**Net Weekly Change:**...",
 "kept": true,
 "finish_reason": "stop"
}

Metric-ladder pair file (1 of 56, DPO→RLVR × chat_gsm8k_train_full at layer 30) — showing the tier structure the aggregate reads: pair_dpo__rlvr_chat_gsm8k_train_full.json @ round-3 mirror. The per_layer.30.raw.tiers block carries t0..t8, each with r2, gap (= within − tier), and a 1,000-draw paired-bootstrap CI on gap; the per_layer.30.baselines.within block carries the identity+bias baseline R² and the kNN retrieval acc-at-k used in the dissociation result. sufficient_tier is computed by the min-tier-within-band rule above.

LMSYS + IF/UF/SFT rows: digest-only — real-user or mixed corpora (content hygiene). No LMSYS or UF prompt/completion text is reproduced here. All 5 models × their respective corpora are at raw_completions/ @ 34333a0bde; seed-42 5-row structural spot check on the GSM8K train RLVR file: all kept=true, finish_reason == "stop", no truncation-length rows in the sample; zero fishy rows.

Verifier acknowledgments: the total Takeaways + Goal + Results prose exceeds the 800-word skim budget (per-result prose band + total-prose budget + figure-caption length + Takeaways bullet-length + paragraph sentence-cap WARN classes acknowledged); the body now carries ten result sections across rounds 3–4 (the round-3 metric ladder plus the round-4 pooled/off-policy/cluster/baseline reads), and the extra content is carried deliberately. The off-policy text-source read deliberately reuses the preceding figure's orange bars and open circles rather than re-embedding the same figure; its per-slice values live in the pooled cell JSONs.

Results

The DPO→RLVR step needs no coordinate change on four of eight corpora (raw) and at most a rotation on seven of eight; the two exceptions are the two corpora RLVR trained on

Each cell is the cheapest correction — tier 0 (source map unchanged) to 8 (linear reparameterization of both sides) — whose gap Δ_t = within-stage R² − tier-t R² falls inside a 0.021-wide window around zero at the paired-bootstrap point estimate; "×" means no tier ≤ 8 suffices. Δ = tier-8 gap point estimate.

Figure. The RLVR step needs no reparameterization on most surfaces. Sufficient tier per (stage-pair × corpus) at layer 30, raw scale. Green = same map (tier 0), yellow-orange = coordinate change, red = no tier ≤ 8. Base rows red (Δ_tier8 +0.14 to +0.26); DPO→RLVR green on 4/8, tier 2–5 on 3/8, red on 1/8. GSM8K test (n=1,319) is an n<d companion column.

Stage-pair	Sufficient-tier distribution across 8 corpora (raw)	Max Δ_tier8 (raw)
base → SFT	none × 8	+0.228 (MATH)
base → DPO	none × 8	+0.260 (MATH)
base → RLVR	none × 8	+0.253 (MATH)
base → longer RLVR	none × 8	+0.263 (MATH)
SFT → DPO	t5 × 1, t7 × 2, t8 × 3, none × 2	+0.043 (IF)
DPO → RLVR	t0 × 4, t2 × 1, t3 × 1, t5 × 1, none × 1	+0.025 (IF)
DPO → longer RLVR	t3 × 1, t5 × 3, t8 × 2, none × 2	+0.116 (IF)

The two DPO→RLVR non-null cells are MATH (rotation at t5, Δ_tier8 +0.008) and IF-constraints (Δ_tier8 +0.025 escapes the band) — the only two panel corpora RLVR trained on. On the recal companion (a per-dim affine rescale, 4,096 free parameters, absorbs rotation/reparameterization partially), DPO→RLVR closes at tier 0 on 6/8. Raw disagrees with recal on 3/8 (GSM8K test raw t2 → recal t0; GSM8K train t3 → t0; IF-constraints "none" → t8), so raw is binding and recal is the companion the rescale accommodates.

The base map never reconstructs a post-trained map; the SFT step installs the map, and every later step is a coordinate change

Mean across 8 corpora of the source-map R² at each of the 9 tiers, layer 30, raw pooled R². The four base→⟨stage⟩ pairs start at t0 R² between −0.52 (base→SFT mean) and −1.59 (base→RLVR mean); per-corpus t0 floor is −4.56 (base→RLVR × MATH). They climb to R² 0.33–0.36 at t8, still well below the within-stage ceiling ≈ 0.55.

Figure. Stage-pair R² recovery curves separate cleanly. Mean R² across 8 corpora per tier (t0 direct → t8 full linear reparam), layer 30, raw; bands ±1σ across corpora. base→⟨stage⟩ pairs rise from R² between −0.5 (base→SFT) and −1.6 (base→RLVR) at t0 to R² 0.34–0.36 at t8. DPO→RLVR and DPO→longer-RLVR sit above R² 0.42 at every tier and are nearly flat. SFT→DPO starts at R² 0.27 at t0, dips below zero on 2/8 corpora (GSM8K test, MATH), and climbs to ≥ 0.55 by t5.

The three post-training pairs start at very different t0 means — SFT→DPO 0.27 (2/8 corpora below zero: GSM8K test −0.09, MATH −0.18), DPO→RLVR 0.51 (all 8 positive), DPO→longer 0.43 (all 8 positive). No linear reparameterization of the base map reconstructs any post-trained map at t ≤ 8; a small linear correction of the DPO map often suffices for RLVR. Post-training installs the map at SFT; the Tülu-3 RLVR step moves the coordinate system less than DPO did on 7/8 corpora.

The per-corpus Δ_tier8 view: RLVR training surfaces cluster near zero for DPO→RLVR

Per (stage-pair × corpus) Δ_tier8 with 1,000-draw paired-bootstrap 95% CI, layer 30, raw pooled R² — the low-level per-unit data behind the aggregate. Values inside the 0.021-wide band around zero mean the source map (fully reparameterized at t8) matches the within-stage fit inside the band; above the band means it cannot.

Figure. Post-training pairs cluster near zero; base pairs sit at +0.14 to +0.26. One point per (stage-pair × corpus) with 1,000-draw paired-bootstrap 95% CI, grey band = 0.021 window. DPO→RLVR at-or-inside on 7/8 (IF-constraints at +0.025 escapes); base pairs cluster in a 0.14–0.26 strip (upper end +0.263 = base→longer × MATH; +0.260 = base→DPO × MATH). GSM8K test (n<d) enters as a companion point.

The headline C_v2 = Δ_tier8(dpo→rlvr, GSM8K train, chat) = −0.0031, bootstrap 95% CI lower −0.0039 / upper −0.0024 — signed CI excluding zero, magnitude 5× smaller than the 0.021 band. Reparameterized-DPO reconstructs the RLVR map on GSM8K train marginally better than the within-RLVR fit does on the same corpus (direction of effect, not "matches to zero"). The two RLVR-mix corpora with positive DPO→RLVR gaps are IF-constraints (+0.025) and MATH (+0.008 at t8 rotation-sufficient); GSM8K, UF, Tülu-3 SFT-mix, and LMSYS × 2 renders all close at t0 or a low tier.

Pooled across seven corpora, map strength is installed at SFT and the DPO→RLVR increment is the smallest step on the ladder

Round 4 pools the seven chat-render corpora into one group-aware fit per checkpoint and training-text arm: pooled held-out R² at layer 30 on the 20% test partition, blue bars fit on the checkpoint's own sampled text (on-policy), orange on the four other checkpoints' text concatenated (off-policy), open circles slicing each off-policy fit by which checkpoint wrote the text.

Figure. Map strength rises at SFT and barely at RLVR. On-policy pooled R² (blue): base .413 → SFT .578 → DPO .598 → RLVR .602 → longer-RLVR .619. Off-policy fits (orange) sit at .45–.46 for every post-trained checkpoint; the open circles show the base-written text slice is the outlier in every post-trained cell.

On its own text the pooled map strengthens at every step only on the frozen test-partition convention: both companions put the DPO→RLVR step at ≈ −.002 (CV .5935 → .5916; matched-n .5619 → .5600) — the sign flip strengthens "barely moves" and refutes strict monotonicity. Two Methodology qualifiers attach to every pooled number: the fold-reproducibility gate formally failed and was adjudicated as instrument conservatism the pooled values inherit (roughly .05–.07, certified only on the LMSYS slice — never compared numerically against round-3 fits), and the ordering is stated at the frozen layer 30, where the max-over-layers ordering differs. The test side excludes 306 near-duplicate-quarantined rows (0.64%).

Post-trained maps transfer across each other's text but not the base model's; the base-text slice is the outlier in every off-policy cell

This result reads the orange bars and open circles of the figure above: the off-policy pooled R² per activation checkpoint, and the same fit's test-side R² sliced by which checkpoint wrote the answer text. Every post-trained checkpoint loses .13–.16 R² when fit on the other checkpoints' text (.45–.46 vs .58–.62 on its own), but the loss is not uniform: text written by SFT, DPO, RLVR, or longer-RLVR is predicted at .52–.57 from any post-trained checkpoint's activations, while base-written text is predicted at .13–.19 everywhere.

The base cell inverts the pattern — its own sampled text is its worst slice (.413 on-policy vs .554 off-policy). The outlier therefore tracks the text, not the checkpoint, consistent with base-generation degeneracy (the round-1 generation audit put the base LMSYS keep rate at 0.63 with a 21% truncation-flag rate). The answer-text divergence diagnostic that would separate text degeneracy from genuine map change was not run (Methodology limitations), so this read is conditional on unmeasured divergence.

The pooled base-map gap is largely a text effect: out of band on own text, four-fold smaller on fixed target text

Per corpus (columns) and training-text arm (rows): the base checkpoint's pooled map transferred to each post-trained stage — own-map ceiling, direct transfer, context-side reparameterization (tier 6), both-sides reparameterization (tier 8); held-out R² at layer 30.

Figure. The base map cannot be reparameterized into any post-trained map on own text; on fixed target text it nearly can. Top row (on-policy): both-sides reparameterization (red) stays .12–.28 below the own-map ceiling (blue; IF-constraints base→longer +.45). Bottom row (off-policy): the same correction closes to within .03–.14 of the ceiling (IF-constraints base→longer +.22).

On the models' own text the pooled read replicates round 3: no linear correction of the base map reaches any post-trained ceiling (tier-8 gaps +.12 to +.28; base→longer-RLVR on IF-constraints an outlier at +.45). On fixed target text the same gaps shrink to +.03–.14, the same cell the outlier (IF-constraints base→longer +.22) — much of the base map's on-policy irrecoverability rides on the text distribution each checkpoint writes.

The GSM8K-train and GSM8K-test panels track each other at every stage (RLVR on-policy .378 vs .384), so RLVR shows no advantage on the exact prompts it trained on; censoring on that contrast is bounded at 0.23% (Methodology). MATH and GSM8K-train carry the heaviest cross-corpus topical entanglement (merged mass .785 and .709), the backdrop for per-corpus reads there.

Direct transfer suffices for DPO→RLVR on most pooled slices; the residual sits on RLVR's own training mix

Each cell of the transition × corpus grid is colored by the cheapest correction class — direct transfer, context-side, both-sides reparameterization, or none within the band — that closes the pooled transfer gap at layer 30; cell text is the tier-8 gap.

Figure. DPO→RLVR closes cheaply nearly everywhere; base rows close nowhere. On own text, direct transfer or a context-side correction closes DPO→RLVR on six of eight columns and the both-sides correction closes MATH; IF-constraints stays open (tier-8 gap +.02). Every base→stage cell is out of band in both arms.

The pooled classification reproduces the round-3 ordering: base rows never close; SFT→DPO closes only with the both-sides correction; DPO→RLVR closes within the band on seven of eight columns. The residual sits on RLVR's own training mix — IF-constraints is the one open DPO→RLVR slice (own fit .514 vs .498 after full reparameterization), and MATH needs the both-sides correction. Transitions into longer-RLVR re-open the grid (IF-constraints tier-8 gaps +.20 to +.26 on own text), repeating round 3's dose pattern.

Counting UltraFeedback and SFT-mix as one quasi-unit (71% of cross-corpus near-duplicate edges join them, so the two slices are correlated), the count is six of seven.

Per-slice tier-8 gaps separate into two strata, with IF-constraints the recurring outlier

The per-unit data behind the classification above: one point per transition × corpus × training-text arm — the tier-8 gap (own-fit R² minus fully-reparameterized-source R² at layer 30), with the 0.021 equivalence band dashed.

Figure. Two regimes, one outlier corpus. Base-pair gaps sit at +.12 to +.28 on own text (off-policy open circles +.03 to +.14, IF-constraints base→longer +.22); post-training pairs cluster at or inside the 0.021 band. IF-constraints (purple) is the recurring escape: +.45 at base→longer-RLVR, +.20 to +.26 on every post-training transition into longer-RLVR.

The point cloud shows the classification is not an artifact of class boundaries: post-training transitions form a tight cluster at the band, base transitions a separated stratum an order of magnitude higher, and IF-constraints is the single recurring outlier — consistent with round 3's finding that the DPO→RLVR residual lives on the corpora RLVR trained on. Off-policy gaps run uniformly smaller than their on-policy siblings, the same text-distribution effect as the base-map grid above.

No cluster's improvement escapes the selection-matched null on own text at any transition

Per adjacent transition and arm, each point is one prompt cluster from the pooled split: its delta-Q — the mean over its held-out prompts of the per-prompt change in R² from source to target checkpoint under the pooled context-side read — against cluster size. Dashed: the 97.5% band of the max-cluster statistic under a within-corpus block-permutation null (1,000 draws, selection applied inside each draw); dotted: the achievable ceiling.

Figure. Nothing localizes on own text. On-policy (top row): the most-improved cluster sits below the null band at every transition (p = .14, .08, .39, .17); bands sit well below ceilings, so the test is informative. Off-policy (bottom row): three of four transitions reject, but the top clusters never match across arms.

Every on-policy transition is a failure-to-reject: the most-improved cluster is indistinguishable from the null given the variance — noise-limited, not evidence that no cluster moved. The off-policy arm rejects on three of four transitions, but the top-three clusters never overlap between arms, so no cluster-level teaching story is corroborated across arms; and the permutation null tests within-corpus exchangeability only, which topical heterogeneity already violates, so a rejection is weak alone. The regression-to-the-mean check on top clusters is moot (no cluster survives); the cluster-resampled sensitivity CI is unrun (Methodology).

The fitted map and identity+bias dissociate: ridge wins held-out R², identity+bias wins rank-1 retrieval in nine of ten pooled cells

Held-out R² (left half of the figure) and cosine rank-1 retrieval accuracy (right half) for the fitted ridge map versus the identity+learned-bias baseline — five pooled on-policy cells, layer 30; retrieval carries 95% Wald intervals and the chance line.

Figure. Ridge wins R²; identity+bias wins retrieval. Left: identity+bias sits at R² −1.21 to −3.46 against ridge's +.41 to +.62. Right: identity+bias beats ridge at rank 1 on every on-policy cell (RLVR .615 vs .602 cosine, chance .0005); the ridge trajectory keeps the SFT jump and the flat DPO→RLVR step.

Identity+bias wins rank 1 in nine of ten pooled cells (exception: base off-policy, ridge .534 vs .506 cosine) yet loses held-out R² everywhere — the raw context activation already carries the neighborhood structure, so map-strength claims here are R²-specific. Ridge retrieval still replicates the stage ordering, so identity winning rank 1 does not undercut "SFT installs the map".

The round-3 per-corpus cells repeat the shape under the earlier fit regime (identity+bias −0.95 to −3.41; kNN acc@1 .38–.86, chance ≤ .001; per-cell values in the pair files), not directly comparable to pooled values — so the tier agreements compare two strongly discriminative maps, not vacuous ones.

The block below is not a sample of model outputs — it is 2 example superseded round summaries preserved for provenance, sanitized for context hygiene.

Rounds 1–2 findings (superseded by round 3)

Round 1 (5,000-prompt corpus at recalibrated scale) headline C = gap(RLVR) − gap(DPO) = −0.00003 with a bootstrap 95% CI running from −0.00237 to +0.00223 on GSM8K train chat. Round 2 (inline dedup + fixed-layer null) left the headline dedup-invariant (zero duplicate GSM8K prompts) and the resume verdict intact (0.237 → 0.231 vs bar 0.201). Round 3 raises the corpus scope by 3.8× on GSM8K (7,473 vs 5,000 kept, actually 4,903 kept), 3× on LMSYS (~15,000 vs ~2,900 kept), and adds four stage-matched corpora that did not exist in round 1 (MATH, IF-constraints, UF, Tülu-SFT mix). Round 1's estimator required a per-dim held-out affine recalibration to rescue LMSYS reads from an n_train < d artifact (raw pooled R² −0.93 kill → +0.24 recalibrated). Round 3's fatter fold-train (2–3× d) + wider λ grid + inner-group-CV selector plus the primal d-space fold path deliver raw pooled R² 0.44–0.61 across every corpus — no rescue needed. The rounds-1–2 kill-and-recover saga is a validation of the recalibration remedy, not an ongoing headline concern; the plan-registered raw scale is primary in round 3.

Round-1's adjacent-increment read on GSM8K train (base→SFT +0.139, SFT→DPO −0.0055, DPO→RLVR −3e-5) is subsumed by round-3's metric-ladder read: base→SFT tier-8 gap +0.186 (raw) across all 8 corpora is orders of magnitude above the band; SFT→DPO closes at t5–t8 on 6/8 corpora and "none" on 2/8; DPO→RLVR closes at ≤t3 on 6/8 corpora. The one-shot SFT effect and the near-zero DPO→RLVR contrast both replicate at the wider scope. Round-1's λ-edge and matched-n figures (figures/issue_1336/lambda_floor_degeneracy.png, matched_n_comparability.png) documented estimator regimes that round 3 largely resolves; they remain committed at 11daecb303 but are no longer load-bearing for the headline.

---

Repro: compute ≈ 214–216 GPU-h total across three rounds — round 1 wave-1 generation + capture + fits on a 4× H100 RunPod pod (pod-1336; ~19–21 GPU-h), round 2 diagnosis + resume + recalibration on GCP A100-80 + a cpu-mid instance (~0 GPU-h), round 3 full-corpora + metric ladder on the Charmander H200 fellows lane (8-GPU node, 194.37 GPU-h vs 98 budgeted — a 2.0× compute deviation carried as a caveat) · code SHA rounds 1–2: 11daecb303 (branch issue-1336; dedup + increments figures at bb7b11c983 and a47359d5c3); round 3: 59ef62aaa9 (branch issue-1336-fullcorpora) with figures pinned to b9ef76654c · round-1–2 eval JSONs: eval_results/issue_1336/ (decision/headline_contrast.json, ladder_alignment/, cells/, diagnosis/recal/, gates/); round-3 v2 artifacts uploaded to HF at issue1336_rlvr_ladder/eval_results_mirror_v2/ (metric_ladder/ with 56 pair files, cells_v2/ with 120 files, gates_v2/ incl. g0v2.json, g1v2_gate.json, g2_capture_parity.json, ladder_pilot.json, v2_bars.json) · figures: rounds 1–2 at figures/issue_1336/ @ a47359d5c3; round-3 hero + supporting at figures/issue_1336/ @ b9ef76654c · raw completions + turnstores + prediction matrices: rounds 1–2 at issue1336_rlvr_ladder/ @ 8c54f9fc; round-3 at issue1336_rlvr_ladder/ @ main (analysis tensors + corpora_v2 + eval_results_mirror_v2) · Reused prompts + gate anchors from #825: the 5,000-prompt Track-S list track_s.jsonl + Qwen instruct/pretrained activation shards @ deb7a452 — the round-3 fit driver re-ran the #825 Qwen anchor fit two ways: under the parent's legacy fit settings it reproduces the committed 0.6731 anchor within abs_dev 5.9e-6, and under the round-3 selector it refits to s_qwen_v2 = 0.6935 (primal-vs-Gram agreement 2.2e-16), whose ratio to the committed anchor sets ex_v2 = 1.0303 · GSM8K: openai/gsm8k @ 740312add88f · Round 4 (pooled-multidataset-onoff-policy-stage-transfer): 2,126.15 GPU-h recorded vs 210 budgeted (10.1×, fellows H200 lane — the figure aggregates every round-4 attempt, including the over-running off-policy capture job and two crashed-then-relaunched fit jobs) · round-4 code SHA 1c918a6d6f (branch issue-1336-fullcorpora) · round-4 eval JSONs on main at f22b6dc90d: cells_pooled_v3/ (10 pooled cells + matched-n companions), metric_ladder_pooled_v3/ (20 transition pair files), decision_v3/cluster_delta_q_per_transition.json, gates_v3/ (g0v3.json, g0v3_adjudication.json, g1v3_gate.json, g2v2_offpolicy_parity.json) · round-4 figures at 06e841e479 · round-4 analyzer digest (every quoted round-4 number with its source path) at eff921f7d8 · round-4 HF artifacts at analysis_tensors/ @ b2c7ed08f1: preds_pooled_v3/ (pooled prediction matrices — predictions only, no ground-truth answer vectors; the reason the leakage-excluded sensitivity read is unrun), pooled_split_v3/split_manifest.json (group table + near-duplicate audit), metric_ladder_pooled_v3/, delta_q_perdraw/, turnstore_offpolicy_* (off-policy captures), plus the round-4 eval mirror at eval_results_mirror_v3/ @ b2c7ed08f1 · round-4 declared discard: gates_v2/g2_recapture/*.pt capture-parity recapture tensors, regenerable from persisted rollout text (task marker v310) · plans: v3 (approved, rounds 1–2) + v7 (diagnosis) / v9 (recalibration) + v13 (approved, round 3) + v14–v28 (round 4: v17 user-approved; v18–v28 gate and analyzer-instruction amendments) under tasks/<status>/1336/plans/ · WandB: n/a (no training).

Context: Origin prompts (verbatim):

Round 1: Help me to run an issue to test the RLVR part of next steps here: [#825 report — Next Steps: could check if doing RLVR changes the base -> instruct mapping more (the model I'm using is too old to have done RLVR)]

Round 3 (full-corpora-stage-evals-metric-ladder): train on all of them. Train on more lmsys for hte mapping. Also choose other evals that are related to each of the stages to test the R² at each stage. And for reparameterization consider this: [the mapping-similarity metric ladder — committed verbatim at docs/mapping_similarity_metrics.md @ fc1087637a]. And make sure you are choosing lambda well (look at past issues). How long will this all take

The block below is 1 example of run provenance, not a sample of model outputs — the round-4 originating prompt (pooled-multidataset-onoff-policy-stage-transfer), preserved verbatim and in full (it embeds a markdown plot spec with math notation):

Round 4 origin prompt (verbatim)

run this now in the background with happy coder. at the end it should just make this plot:

Methodology

• Model: [model that has all checkpoints]
• Collect context and answer vectors for:
  • generic LMSYS/WILDCHAT (5000 contexts)
  • domain specific evals:
    • insert
  • split combined data into train and held-out eval set
• Fit ridge regression mapping on train set (generic data and domain specific evals) in:
  • base model
  • post SFT
  • post RLHF
  • post RLVR
• See how well each mapping transfers from each stage to each other stage
  • mapping trained on either **on-policy text ** or off-policy text from the source stage
  • mapping evaluated on generic data and each domain specific eval

Results

Result 1: Transfer from each stage to each other stage

Plot showing:

• $R^2$ for the mapping trained at each stage
• $R^2$ for the mapping trained at each other stage directly transferred to that stage
• $R^2$ for the mapping trained at each other stage transferred to that stage (with reparameterization of ONLY context vector)

4 plots:

• 2x2 of on-policy/off-policy mapping, for each evaluation dataset

Lineage: #825 — the Qwen elicitation-not-teaching parent this run extends to RLVR (#779 built the map construct). Created 2026-07-15; round 1 run 2026-07-15/16 (wave-1 launched and killed by the plan-set within-stage strength threshold on 07-15 with raw LMSYS reads −0.93 chat / −0.89 naturalistic, diagnosis 07-15, held-out recalibration + resume decision 07-15/16); round 2 run 2026-07-16 (inline dedup + fixed-L29 null free-analysis, epm:free-analysis-followup-run v1, artifact commit bf5ed3945f); round 3 run 2026-08-02/03 on the Charmander H200 fellows lane (extract phase 2026-08-03 ~03:00–12:26Z; fit phase 12:26–15:33Z; metric-ladder phase 15:33–22:03Z; upload 22:03–22:28Z; epm:results v7 @ 22:28:49Z), interpretation drafted 2026-08-03 by the analyzer in HOLD-marker mode pending upload-verifier PASS; round 4 scoped 2026-08-04 (epm:followup-scope marker), plan v17 user-approved 2026-08-12, run 2026-08-12/13 on the same fellows lane (fold-reproducibility gate fail + adjudication 2026-08-13 01:42–02:26Z, marker v276; off-policy capture + pooled fits SLURM jobs 12422 / 12643 / 12874; epm:results v311 @ 2026-08-13T23:40Z), folded into this clean-result 2026-08-13.