Held-out variance explained and retrieval accuracy dissociate in both directions across the context-to-answer estimator ladder (HIGH confidence)

Methodology: docs/methodology/issue_1901.md · earlier methodology snapshot

Takeaways

• Both dissociation directions realized on both target corpora: identity plus bias scores acc@1 0.53 at pooled R² −0.92 on LMSYS and 0.53 at −0.88 on WildChat (n=1000 each); a shrinkage-limited prefix ridge scores R² +0.54 at retrieval 0.04 (chance 0.02).
• At the 963k training regime, fitted maps beat every baseline on all non-saturated metrics (cosine and median rank saturate); the nonlinear-over-ridge acc@1 gap grows with pool size on both corpora — 0.036 to 0.101 (LMSYS) and 0.025 to 0.079 (WildChat) across pools 1,000 to 100,000. Averaging one to five evaluation answers raises frozen-map R² by 0.052–0.055, but corrected retrieval by only 0.98–1.21 percentage points (942 candidates). Extending five to ten adds 0.0077/0.0064 R² for linear/nonlinear and 0.53/0.21 retrieval points; the nonlinear retrieval interval includes zero.
• A dense within-store training-size ladder (8 fresh rungs, 5,000-500,000 rows, one pinned eval pool) confirms the neural map keeps gaining where ridge plateaus: over 50,000-500,000 rows the width-8192 map's held-out R² rises 0.761 to 0.807 against ridge's 0.744 to 0.761, the slope gap's 95% lower bound is +0.028 against the plan's 0.01 margin, and all 1,000 paired bootstrap draws clear it; ridge leads through 10,000 and the arms cross near 25,000. The published cross-store scaling join is superseded — its two stores' pinned eval pools share 0 of 400 checked rows.
• Every cross-class estimator ordering transfers to 1,000 fresh held-out WildChat targets (held-out-row within the training distribution, context-arm-only): euclidean acc@1 rank agreement is exact with zero inversions (ridge 0.800, wide neural map 0.844, identity plus bias 0.527); pooled-R² agreement is 0.93, its only resolved flips inside the identity-plus-bias companions.
• Instrument and provenance checks hold: shuffled-pair nulls collapse to metric floors on every battery (raw cosine floors at 0.68-0.71 while the constant train-mean scores near 0.8), retrieval hubs are corpus-structured (the correction buys fitted maps 0.03-0.10 acc@1), ridge falls 0.81 to 0.065 acc@1 as training rows drop 963k to 50 while identity plus bias stays near 0.50, and deliberately in-train targets inflate fitted-map acc@1 by +0.07 to +0.11 at R² shifts of at most 0.03; headline layers are inherited (context 19, prefix 18) and layers 14/26 reproduce every ordering.
• A generic-boundary-token control anchors the scaling figure: the WikiText sentence-final "." map reaches R² 0.440 and acc@1 0.344 at 963,444 pairs against the chat map's 0.754 and 0.805, with the gap's 95% lower bound above 0.29 at every matched size from 1,000 up — the fully generic next-span account is ruled out (generic contributions inside the chat map, and corpus or difficulty differences, are not); the "!" and "?" single-type arms are N/A — not tested (14 and 9 eval spans); the prior pooled punctuation-control reading (0.1087) was a fraction-of-ceiling misread (transfer R² 0.0732), superseded.

Goal

• This experiment in context: The standing rule to co-report an identity-plus-learned-bias baseline and a kNN-retrieval read beside held-out R² came from two single-estimator reads: identity plus bias reached acc@1 0.84 at pooled R² −6.5 on the 50-context prefix battery (#722), while the fitted ridge dominated retrieval on the 963k-context map (#779); #1776 verified the banked weights reproduce to 8e-11. This experiment turns those spot reads into one systematic reference — the full estimator ladder crossed with every metric, both mapping arms, pool-size and training-size axes — under the same split, layers, and banked weights (directly comparable). The generic-boundary-token control round sharpens the punctuation control from #825/#931 (pooled boundary types, small n, an estimator later shown degenerate in that regime) into a single-type control measured at the chat ladder's own training sizes and estimator conventions. The dense-ladder round re-measures the training-size axis with 8 fresh rungs inside the #779 963k-context capture store after its planned cross-store identity gate measured that the earlier small-n scaling cells' eval pool shares no rows with that store's pinned split — the published joined ladder is superseded, and the within-store curve is now the training-size evidence.
• Broader narrative: The context-to-answer mapping line treats a context's effect on the model as a predictable vector; every future map-quality claim rests on these metrics, so their constructs, floors, and failure modes need one canonical measurement.

Methodology

Rollout-count follow-up (2026-09-07): Evaluation-target K=1–5 on the frozen Qwen2.5-7B-Instruct layer-19, 963,444-training-context linear and existing nonlinear maps, plus identity+learned-bias. For each K, average metrics over all size-K subsets of the five banked on-policy answer vectors (31 subsets total); all 1,000 original held-out contexts enter R², and a fixed keep-one exact-original-answer-vector policy leaves 942 retrieval queries/candidates (top-1 chance 1/942). Training-derived whitening stays fixed; whitened cosine with two-sided CSLS (neighborhood 10) is primary, with raw Euclidean/cosine and whitened-cosine companions. Use 2,000 paired context bootstraps, seed 190141, re-centering each R² denominator; retrieval intervals condition on the fixed pool and rankings. Fresh-only K=1–4 excludes the original draw from averaging but retains the same context set. The initial K=1–5 analysis performs no refits or generation. The user-requested K=10 extension adds five same-recipe draws per context (seeds 47–51, engine seed 42; temperature 1.0, top-p 0.95, 1,024-token cap) and compares their union with the exact existing K=5 target using the original bootstrap counts. It also scores the new five alone as a descriptive batch diagnostic. All 5,000 new vectors completed; both 32-row recapture checks have zero relative error. This is a direct K=5 vs K=10 comparison, not exhaustive subsets of ten. Training-target K and K>10 remain untested. K=10 protocol and validation. Protocol, exact inputs, and validation.

• Design: Evaluation-only battery; the manipulated variable within each read is the estimator. Context arm: 14 estimator rungs (7 fitted or baseline estimators at 963,444 training rows, 5 companions at 3,600, 2 at 50) crossed with 8 metrics (pooled R², per-dimension R², mean cosine, kNN acc@k under euclidean and cosine distance, hubness-corrected acc@k, median rank, mean reciprocal rank, hubness diagnostic), 3 layers (19 headline; 14/26 exploratory), and 4 candidate pools (1,000 / 5,000 / 20,000 / 100,000). Prefix arm: 7 estimators over the 50-context battery (7 prefix families), leave-one-family-out. Bootstrap CIs use n=1000 shared test-row resamples; every metric gets a 200-draw shuffled-pair null per arm. Both mapping arms run (context-based and prefix-based); the prefix arm runs at n=50 because no larger prefix-level mapping data is persisted — a stated deviation carried as a scope caveat. Follow-up round wildchat-target-battery (plan v7): one manipulated variable — the test-target corpus (LMSYS to WildChat); banked weights, battery, pools, nulls, and bootstrap conventions inherited verbatim with no refit. The round is context-arm-scoped (the prefix battery is a constructed-prefix design with no WildChat analogue — a stated deviation). Target assembly: 2,500 fresh WildChat-1M candidates streamed past the parent capture frontier at a pinned corpus revision, screened against a 956,693-fingerprint training-plus-held-out exclusion set and a transposed 965,000-row train-pool near-duplicate gate (5-gram Jaccard 0.8; 521 dropped, 1,979 survivors, first 1,300 kept); captured through the parent rig on one fellows-cluster GPU (1,299 of 1,300 rows captured; 1 over-length skip recorded in an uploaded sidecar); the first 1,000 captured rows in seeded order are the test targets, and the battery re-checks every target fingerprint against the exclusion set (0 hits). Before any WildChat number, the round re-applies the banked layer-19 ridge to the pinned LMSYS test-1000 and asserts the banked pooled R² reproduces (realized delta 8.1e-11). The round plan's optional in-train memorization companion (wc_intrain, exploratory and contamination-labeled by design) was deferred from the round itself and then run as the task's single zero-GPU free-analysis follow-up (screened not-redundant): the identical 14-arm battery on 1,000 deliberately in-train WildChat rows sliced deterministically from the banked parent capture chunks (seed offset +1; the contamination re-check reads 1,000 of 1,000 exclusion-fingerprint hits, expected by construction), no refit, layer 19, same pools/nulls/bootstrap; a paired deterministic held-out re-run reproduced the round's published values identically. It is a memorization read — never a headline arm; the transfer headline rests on the held-out arm alone. Follow-up round generic-boundary-token-control (plan v10): one added control arm on the scaling figure — a fixed generic boundary token. Map input = the residual-stream state at a sentence-final "." token in WikiText-103-raw-v1 (teacher-forced forward of real corpus text; no generation anywhere in the round), target = the mean activation of the next span (the text up to the next sentence-final boundary, 8-256 tokens), layer 19 headline with 2/14/26 companions; ridge fits at the chat ladder's verbatim training-size grid (15 common rungs, 50 to 963,444; three seeded draws at n ≤ 2,500) with validation-selected regularization, an article-disjoint eval split (1,000 test / 400 validation spans, seed 42), exact-duplicate plus 5-gram near-duplicate screens of the train pool against the eval spans, and the same 1,000-target retrieval-pool conventions. The realized "." yield after screens is 1,278,986 pairs (1,277,586 train pool + 1,400 eval), covering the full banked grid. The chat-vs-control gap is classified per rung from a conservative interval difference — the banked chat bootstrap interval against an article-level cluster bootstrap on the control side (chat rungs at or below 25,000 carry banked points only, flagged in the artifact). Per-rung baselines on the control pairs: identity plus bias, constant train-mean, and a 200-draw shuffled-pair null (advisory). The "?" and "!" single-type arms and a pooled bridge fit rode the same capture; both single-type arms yielded too few eval spans to score (9 and 14) and are N/A — not tested. Follow-up round mlp-scaling-densify (plan v15): one manipulated variable — training-set size within one store. The round's planned first gate re-checked whether the earlier small-n scaling cells' pinned eval rows are the n1m pinned rows; it measured 0 content matches over all 400 checked validation rows (median nearest-neighbor distance 6.63 against a 2e-3 tolerance), so the round pivoted from densifying the joined ladder to a fully within-store design and recorded a machine-readable supersession record for the join's consumers. Eight fresh rungs (5,000 / 10,000 / 25,000 / 50,000 / 100,000 / 150,000 / 250,000 / 500,000) draw LMSYS rows uniformly without replacement (seeded, banked selection helper, sha-pinned per rung) from the 963,444-row training pool; each rung fits a paired ridge and width-8192 neural map on identical rows against the same pinned 400-validation/1,000-test rows; the two verdict endpoints (50,000 and 500,000) add neural-map seeds 43 and 44. Banked anchors ride the figure as descriptive context outside the claim boundary: the earlier store's prefix-draw 5,000-25,000 cells, the banked 963,444 weights applied to the pinned pool, and refit-parity checks at 150,000 and 500,000. Verdict rule (disjoint and exhaustive): Confirmed ⇔ D_gap = (S_mlp − S_ridge) − 0.01 ≥ 0, where S is each arm's R² rise from 50,000 to 500,000 (neural map averaged over three seeds); the two descriptive sub-reads (both-arms-plateau, ridge-decline-driven) both fired false.

• Training: N/A — no model training in this task. The four fitted maps are banked weight payloads applied unchanged (in-run reproduction assert: applied ridge pooled R² matches the banked 0.7541708417500046 within 1e-6; realized delta 8.1e-11 in the parent run and in the WildChat round's re-assert). Their production recipe, inlined: fit on 963,444 mixed LMSYS+WildChat contexts (pinned split seed 42; 400 validation / 1,000 test rows held out with sha-pinned membership; the training pool near-dupe-screened against the 1,400 validation/test targets at 5-gram Jaccard 0.8 — 435 exact and 30,437 near duplicates dropped), inputs = the last-context-token activation of Qwen2.5-7B-Instruct at layer ℓ, targets = the mean answer-token activation at the same layer, layers 14/19/26. Ridge: streaming fp64 primal regression, 23-point λ grid from 1e-3 to 1e8, validation-selected λ = 0.001. Neural maps: one GELU hidden layer at widths 8,192 and 32,768, MSE loss, AdamW lr 3e-4 with weight decay 1e-4, batch 4,096, validation early stopping (patience 20, epoch cap 300) on an internal seeded 10% early-stop split. Kernel map: RBF kernel ridge, Nyström with 16,384 landmarks, γ at the median heuristic, λ validation-selected at 0.1. The AdamW learning rate 3e-4 is copied from the producing run's committed fit constants and differs from this task's plan by design — the plan trains nothing, so it declares no learning rate. The dense-ladder round's fresh rungs are new fits of the same two recipes (no model training; activation-space regressions only). Complete analysis-knob table (parent rows cite the parent plan; plans/plan.md now points at the round amendment v15):

  Knob	Value	Source
  Battery seed (distractors, nulls, bootstrap)	1901 (all rounds)	plan §10
  Split (train/val/test)	3600 base + 959,844 new / 400 / 1000, seed 42, sha-pinned	banked split metadata (n1m_fits.json)
  Layers	19 headline (validation-selected upstream); 14, 26 exploratory	banked fit metadata
  Banked ridge recipe	streaming fp64 primal; λ grid 23 points 1e-3 to 1e8; selected λ = 0.001	banked fit metadata + docs/methodology/issue_779.md
  Banked neural-map recipe	1 GELU hidden layer, widths 8192 / 32768; MSE loss; AdamW lr 3e-4, wd 1e-4; batch 4096; val early stop, patience 20, cap 300, internal seeded 10% split	docs/methodology/issue_779.md (fit constants) + issue779_ffc_n1m_fits constants
  Banked kernel recipe	RBF Nyström m = 16384; γ median-heuristic; λ ∈ {0.1, 10}, selected 0.1	banked fit metadata
  Small-n ridge rungs	banked λ = 1000 at n = 3600 and n = 50 (fixed, no re-selection); WildChat round refits on unchanged inputs, only the evaluation targets swap	context_arm.json + wildchat_arm.json small_n_meta
  Retrieval k values	(1, 5, 10) context; (1, 3, 5) prefix	mapping_baselines.knn_retrieval default + banked convention
  Candidate pools	1000 (test) / 5000 (test+train+val rows) / 20,000 / 100,000 (test+distractors)	plan §11 (decade-spaced; bilingual-lexicon evals use up to 200k, arXiv 1710.04087)
  Hubness-corrected score	CSLS, cross-domain neighborhoods, k = 10	arXiv 1710.04087
  Bootstrap	n_boot = 1000, test-row resampling, one shared draw matrix across estimators	banked CI convention (BOOT_N)
  Null	K = 200 shuffled-pair permutations per metric × arm	plan §11 (banked 200-draw null convention)
  Distractor pool	seeded 210 of 1,920 capture chunks streamed; 100,000 layer-19 rows kept	plan §11 + distractor_manifest.json
  Prefix-arm neural rung	batched leave-one-family-out MLP, width 512, 300 epochs	banked battery recipe (vectorized_mlp_skill)
  Prefix-arm headline layer	18 (validation-selected in the banked battery; inherited)	eval_results/issue_722/identity_bias_knn/results.json best_ridge_layer
  WildChat round: target corpus + revision pin	allenai/WildChat-1M @ 7d6490e462285cf85d91eabea0f9a954fbddcd1f	issue1901_wildchat/manifest/meta.json
  WildChat round: candidate screen	2,500 streamed; 956,693-fingerprint exclusion + near-dupe gate (5-gram, Jaccard 0.8); 521 dropped; first 1,300 kept; 8-worker screen, 222.7 s	plan v7 §4 w0 + manifest meta screen
  WildChat round: capture recipe	parent constants verbatim (Qwen2.5-7B-Instruct, seed 42, max model len 8192, gen max tokens 1024, layers 14/19/26); 1,299/1,300 captured	plan v7 §4 w1 (parent capture module constants)
  WildChat round: pools	1,000 targets-only / 5,000 / 20,000 / 100,000 (targets + parent distractor rows); 0 duplicate vectors in every pool	wildchat_arm.json pools
  Boundary round: corpus + revision pin	WikiText-103-raw-v1 @ b08601e04326c79dfdd32d625aee71d232d685c3 (streamed)	issue931_common.WIKITEXT_REVISION (the prior punctuation-control rig)
  Boundary round: anchor token	sentence-final "." single token; realized ids 659 (1,247,403 pairs) / 13 (31,583)	plan-time tokenizer probe + boundary_token_secondary.json per-type meta
  Boundary round: pair constraints	next span 8-256 tokens; previous sentence ≥ 8 tokens; article cap 4,096 tokens; ≤ 48 anchors per article (cap and anchors raised for yield)	issue931_build_pairs Arm-C constants + plan v10 §11
  Boundary round: split + screens	1,000 test / 400 val spans, article-disjoint, seed 42; exact dedup + transposed 5-gram Jaccard 0.8 screen vs eval spans; realized "." yield 1,278,986 kept pairs	plan v10 §4 + boundary_token_secondary.json screens
  Boundary round: fits	chat rung grid verbatim (15 common rungs; 3 draws at n ≤ 2,500, seed formula 19010000 + n×10 + draw); validation-selected λ on the parent grids; layers 19 headline, 2/14/26 companions	scaling_ladder_L19.json draw convention + banked fit cores
  Boundary round: gap decision rule	control-distinct ⇔ gap 95% lower bound ≥ 0.25 at every common rung n ≥ 1,000; generic-driven ⇔ top-rung upper bound < 0.15; else unresolved; control CI = article-level cluster bootstrap (1,000 draws, seed 1901, 117 test articles); gap = conservative interval difference	plan v10 §3
  Boundary round: capture	Qwen2.5-7B-Instruct bf16 (frozen, teacher-forced), layers {2,14,19,26} persisted, length-grouped batch 8; 643 shards	issue931_extract_store (parent capture module constants)
  Densify round: rung grid + seeds	8 fresh rungs 5,000-500,000 (seed 42); neural-map seeds 42/43/44 at the 50,000 and 500,000 endpoints	plan v15 §4 + mlp_scaling_dense_L19.json per_n
  Densify round: row selection	uniform draw without replacement from the train pool's LMSYS rows via the banked selection helper (seeded; selection sha pinned per rung)	issue779_ffc_n1m_fits.select_train + sel_sha256 fields
  Densify round: fit recipes	banked cores verbatim — ridge streaming fp64, 23-point λ grid (selected λ: 3162 at rungs ≤ 25k, 1000 at 50k-150k, 316 at 250k, 0.001 at 500k — the grid's low edge, matching the banked big-N cells); neural map width 8192, MSE loss, AdamW lr 3e-4, wd 1e-4, batch 4096, val early stop on an internal seeded 10% split, patience 20 (realized 47-74 epochs vs cap 300); recipe fixed across rungs (val-selected once at L19 upstream, not retuned per cell)	mlp_scaling_dense_L19.json meta.recipe + per_n fit_meta + meta.recipe_caveat; MSE loss + seeded 10% split: plan v15 §11 (fit_mlp)
  Densify round: gates	refit-vs-banked parity on sha-exact inputs (deltas ≤ 4.4e-16); banked-963,444 apply parity (neural map 7.2e-7, ridge 8.1e-11); statistical parity at the endpoint rungs (deltas ≤ 4e-4); cross-store identity gate refuted by measurement (0/400 content matches)	mlp_scaling_dense_L19.json gates + per_n parity
  Densify round: verdict bootstrap	paired context bootstrap over the 1,000 pinned test rows, one shared draw matrix, n_boot = 1000, seed 1901, zero refit from persisted fp16 predictions; slope-gap 95% CI 0.0278 to 0.0329; D_gap 95% CI 0.0178 to 0.0229; 1,000 of 1,000 draws ≥ 0; point recompute worst delta 2.5e-6	dgap_context_bootstrap.json
  Densify round: retrieval convention	whitened-cosine + CSLS primary (train-pool whitening, shrinkage 0.1, k = 10) with raw euclidean companion; 0.94 duplicate acc@1 ceiling shared with the parent battery	plan v15 §6 + context_arm.json dup stats
  Compute env	VM CPU only (parent + free-analysis rounds); boundary round on RunPod pods; densify round on one 1× H100 RunPod pod with the fold's bootstrap on the VM; 8 BLAS threads; MALLOC_ARENA_MAX=2; numpy 2.2.6, torch 2.8.0	context_arm.json + wildchat_arm.json + boundary_token_scaling_L19.json + mlp_scaling_dense_L19.json metadata

• Evaluation: All dependent variables are the mapping metrics themselves, computed deterministically over banked activations — no LLM judge; new generation is limited to the WildChat target capture and the K=10 follow-up's 5,000 additional answers. Per metric: pooled R² = 1 − SS_res/SS_tot on the held-out set (variance-weighted across dimensions; unbounded below); per-dimension R² (exposes the variance weighting); mean cosine between prediction and target (per-row scale-invariant; high anisotropy floor); kNN acc@k = the probability the true answer vector is within the k nearest pool neighbors of the prediction (rank-based, mid-rank tie handling; analytic chance k/n_pool); the CSLS hubness-corrected variant of acc@k; median rank and mean reciprocal rank; and a hubness diagnostic (skewness of 10-occurrence counts, with top-hub corpus composition). Correctness asserts: banked-ridge reproduction within 1e-6 (run in the parent battery and re-run in the WildChat round before any WildChat number), realized payload keys for all 12 weight files, helper-parity checks per retrieval cell, and 0 duplicate groups in the distractor pool. The WildChat round's transfer read: per metric, rank agreement of the 14 common arms between the banked LMSYS values and the WildChat values (Kendall τ), plus a pairwise-inversion table; CI resolution (within-corpus paired-difference bootstrap CIs on shared draws) covers the two headline metrics — pooled R² and euclidean acc@1 — and the 6 remaining point inversions (mean cosine, cosine acc@1, CSLS acc@1, two on mean reciprocal rank, median rank) are near-tie or companion-rung pairs left unresolved; each arm is scored against its own corpus's targets by design — the read is rank transfer, not a cross-corpus value comparison. Known metric ceilings that bound interpretation: 58 of the 1,000 LMSYS test targets have an exact in-pool duplicate vector, capping acc@1 near 0.94 equally for every estimator (ordering unaffected, resolved by k=5), and the LMSYS 5,000-row pool carries 354 excess duplicate rows with the same equal-across-arms effect; the WildChat pools carry 0 duplicate vectors, so this ceiling does not bind there; median rank saturates at 1 for every decent map at pool 1,000. The boundary-control round adds no new metric: per rung it reports the same held-out R² and euclidean acc@1 over its own 1,000 held-out WikiText spans (chance 0.001), with the control-side interval computed at article grain (117 test articles; eval spans cluster within articles) and the chat side read from the banked cells; a rung-50 fit run on both the GPU and CPU paths matched with a delta of exactly 0 (tolerance 1e-6), and the figure regression gate reproduced all 304 previously committed scaling-figure points exactly while adding the 120 control points. The dense-ladder round also adds no new metric: per rung it reports the same pooled held-out R² and retrieval acc@1 (whitened-cosine-plus-CSLS primary, euclidean companion) over the pinned 1,000 test rows. Its verdict quantity is the paired slope gap between the ridge and neural-map 50,000-to-500,000 R² rises minus a 0.01 margin; the realized value is +0.020 (Confirmed), and the fold's paired context bootstrap (one shared draw matrix, zero refit from the persisted fp16 test predictions; committed values reproduce to 2.5e-6) puts every one of 1,000 draws above the margin. Analysis-concern dispositions for that round: the 500,000-row ridge selects its regularizer at the grid's low edge (matching the banked big-N cells; every other rung selects interior values), so the top-rung ridge read is grid-consistent rather than changed by the round; realized neural-map epochs (47-74, cap 300) rule out an early-stop-truncation account of either slope; the training pool was already near-duplicate-screened against the 1,400 evaluation targets upstream, so a duplicate-clean slope is the headline slope by construction; endpoint seed dispersion (standard deviation 0.0009) bounds fit noise, not draw noise — each rung is a single draw; banked-vs-fresh value gaps at the shared 5,000-25,000 sizes (at most 0.009 R²) are attributed to the whole store difference (draw scheme, capture batch, eval pool), not any single knob; and the seamless-curve reading stops at 500,000 — the 963,444 point is the banked weights applied, not a fresh rung.

• Data extraction: Context arm: real-user single-turn conversations from LMSYS-Chat-1M and WildChat-1M (realism tier 1); answers were generated on-policy by Qwen2.5-7B-Instruct in the banked capture, and activations recorded by teacher-forced re-forward. This task adds the distractor pool: a seeded random subset of 210 of 1,920 capture chunks, streamed one chunk at a time (peak footprint one chunk), keeping 100,000 layer-19 answer vectors (55,686 LMSYS / 44,314 WildChat; 0 duplicate vectors in-pool; disjoint from the 5,000 fit-round rows by construction) — uploaded to the HF data repo before analysis. The 5,000-row LMSYS pool reuses rows that were near-dupe-screened against the test targets upstream, so retrieval there is easier than at the unscreened 20,000/100,000 distractor pools, which are the more natural read. WildChat-round targets: fresh real-user WildChat-1M conversations (realism tier 1) verified absent from the training pool (0 exclusion-fingerprint hits over the 1,000 targets at battery time); answers generated on-policy by the same capture rig and activations recorded by teacher-forced re-forward, exactly as the banked capture. No WildChat user text is inlined in this body — WildChat is an unscreened real-world corpus; rows are referenced by file and count only (3 raw-completion chunk files plus 1 over-length-skip sidecar under the round's HF prefix). Boundary-control pairs: real encyclopedic text (WikiText-103-raw-v1, realism tier 2 — an established corpus at a pinned revision); no completions are generated anywhere in that round — activations come from teacher-forced forwards of the raw corpus text; kept "." spans run 8-256 tokens (mean 32.2, median 29). Dense-ladder rows: the same banked capture's activations — the 8 fresh rungs draw LMSYS rows only from the 963,444-row training pool, and no new text, generation, or judging enters the round. Prefix arm: the inherited 50-context battery (families: persona, behavior, format, in-context-learning, rephrase, WildChat, default assistant — constructed conditions, realism tier 3, a scope caveat), prefix-level last-input-token activations and mean answer summaries, query-averaged.

• Sample training/evaluation data + completions: The task's new completions are the WildChat target capture and the K=10 follow-up's 5,000 answers, whose raw text remains in the linked HF artifacts; the worked examples below are verbatim artifact rows. Disclosure: 1 of 448 context-arm retrieval cells, chosen as the headline cell (ridge, test pool, euclidean); full file: context_arm.json.

  {"acc_at_k": {"1": 0.805, "5": 0.896, "10": 0.945},
   "chance_at_k": {"1": 0.001, "5": 0.005, "10": 0.01},
   "median_rank": 1.0, "mrr": 0.8502544488804233, "n": 1000, "n_pool": 1000,
   "helper_parity": "PASS",
   "null": {"acc1_mean": 0.000915, "acc1_p975": 0.003,
            "mrr_mean": 0.007346965163888691, "median_rank_mean": 501.99125},
   "acc1_ci": {"lo": 0.777, "hi": 0.83},
   "mrr_ci": {"lo": 0.8291832391978571, "hi": 0.8701081979849409}}
  

  Disclosure: 1 of 9 per-metric characterization entries (pooled R², the headline metric); full file: metric_characterization.json.

  {"construct": "fraction of held-out target variance the map explains, variance-weighted across dims",
   "computed_as": "1 - ||Y-Yhat||^2_F / ||Y-Ybar||^2_F, SS_tot on the test set's own mean (banked whole_map_r2 convention)",
   "invariances": "sensitive to scale AND offset; dominated by high-variance dims; unbounded below",
   "constant_predictor_score": -0.044533745181768225,
   "empirical_null_floor": {"mean": -0.7618102512485427, "p975": -0.7350515170534669},
   "failure_modes": [
     "a context-independent shift scores catastrophically negative while retrieval stays high (identity+bias)",
     "variance-weighting hides dims the map never moves",
     "a regularization-limited fit can score pooled R2 >> 0 from between-group/mean structure while per-context retrieval sits at chance (this run: prefix ridge-LOFO pooled R2 +0.541 at acc@1 0.040, chance 0.02, with all 7 prefix families at family-level R2 <= 0; context ridge n_train=50 fixed-lambda R2 +0.384 at acc@1 0.065)"],
   "banked_dissociation": "identity+bias R2 -0.865 at medrank 1 (#779 identity_bias_knn); acc@1 0.84 at pooled-OOF R2 -6.5 (#722)"}
  

  Disclosure: summary fields of the 1 distractor-pool manifest (chunk names elided); full file: distractor_manifest.json.

  {"chunk_universe": {"n_total": 1920, "sha256": "cdb00a9cc38808a02dd9fee032002a1abadee3be7930dd5921486d7f2dbca515"},
   "rows_per_chunk": {"n_streamed": 104983, "min": 493, "median": 500.0, "max": 500},
   "corpus_counts": {"lmsys": 55686, "wildchat": 44314},
   "dup_stats": {"n_rows": 100000, "n_unique_vectors": 100000, "n_duplicate_groups": 0, "n_excess_duplicate_rows": 0}}
  

  Disclosure: the headline rank-transfer entry (euclidean acc@1) plus the target-provenance fields, verbatim from the WildChat round's transfer file (per-arm value dictionaries elided); full file: transfer_comparison.json.

  {"kendall_tau": {"acc1_euclid": {"tau": 1.0, "p": 7.740804583966695e-07}},
   "pairwise_inversions": {"acc1_euclid": []},
   "setup": {"wc_targets": {"n": 1000, "n_available": 1299, "in_train": false,
             "contamination": {"n_rows": 1000, "n_exclusion_hits": 0, "expect_hits": false}}}}
  

  Disclosure: 2 of 15 boundary-round gap-interval rows — the binding top common rung, whose chat side carries the two-sided banked bootstrap interval, and the 1,000-pair rung, whose chat side is banked-point-degenerate (as at every rung through 25,000); per-draw fields elided, verbatim; full file: boundary_token_scaling_L19.json.

  {"n_train": 963444,
   "chat_ci": {"point": 0.754170841830715, "lo": 0.7441307847286448, "hi": 0.7638445771747455},
   "chat_ci_source": "banked-bootstrap-ci",
   "control_article_ci": {"point": 0.44029790304000194, "lo": 0.4254366080155413,
                          "hi": 0.4516397922390156, "n_draws": 1},
   "gap_lo": 0.2924909924896292, "gap_hi": 0.33840796915920424}
  

  {"n_train": 1000,
   "chat_ci": {"point": 0.6033319349246442, "lo": null, "hi": null},
   "chat_ci_source": "banked-point-degenerate",
   "control_article_ci": {"point": 0.2705616316164574, "lo": 0.248943808958899,
                          "hi": 0.285665389378687, "n_draws": 3},
   "gap_lo": 0.3176665455459572, "gap_hi": 0.3543881259657452}
  

  Disclosure: 1 of 8 dense-ladder rungs (the 500,000-row rung, the verdict's upper endpoint); nested per-seed blocks, cosine-variant sub-dicts, and secondary fields elided, remaining values verbatim; full file: mlp_scaling_dense_L19.json.

  {"n_train": 500000, "source": "n1m", "sel_name": "lmsys_500k",
   "sel_sha256": "f9ab1707b23d0677d25eae9ce41fce1559346728e819ee8bdc2baaa90c5c55fe",
   "selection": {"mode": "lmsys", "n_target": 500000, "n_realized": 500000,
                 "n_lmsys": 500000, "n_wildchat": 0},
   "parity": {"mlp": {"got_r2": 0.8072743338219446, "banked_r2": 0.8074943555084623,
                      "tol": 0.02, "kind": "statistical", "pass": true},
              "ridge": {"got_r2": 0.7609049151916742, "banked_r2": 0.7609049151916738,
                        "tol": 0.02, "kind": "sha-conditional", "pass": true}},
   "mlp": {"test_r2": 0.8072743338219446,
           "test_ci": {"r2": {"point": 0.8072743338219446, "lo": 0.7972258050262349, "hi": 0.8172818683672245}},
           "whitened_csls": {"acc_at_k": {"1": 0.898, "5": 0.958, "10": 0.995}, "k_csls": 10},
           "knn": {"euclidean": {"acc_at_k": {"1": 0.844, "5": 0.926, "10": 0.975, "50": 0.993}, "median_rank": 1.0}},
           "fit_meta": {"width": 8192, "lr": 0.0003, "epochs_ran": 50, "batch": 4096}},
   "identity_bias": {"test_r2": -0.8962630398928697}}
  

  Conciseness: after the WildChat, memorization, boundary-control, and dense-ladder folds the total Takeaways+Goal+Results prose runs over the word budget, all six Takeaways bullets exceed 30 words, and several per-result blocks sit in the 120-180-word band (total, bullet, and per-result WARNs all acknowledged) — the estimator-ladder results carry the summary-plus-per-unit figure pair the aggregate-result rule requires, the memorization result's dumbbell is itself the per-estimator view, the boundary-control result pairs the scaling summary with a per-cell layer-and-draw companion, the dense-ladder result pairs its scaling summary with the per-rung paired-delta and store-seam companions, and the dense bullets hold the numbers the claims need.

Results

Variance explained and retrieval rank the estimator ladder in opposite orders

One point per estimator and training regime: pooled held-out R² (horizontal, symlog) against retrieval acc@1 (vertical; euclidean; 1,000-candidate pool, n=1000 test rows, context arm; 50-candidate, prefix arm; marker = regime). The grid companion covers all estimators on six metrics.

Figure. The two metrics disagree in both directions. Identity plus bias (green) sits top-left: high retrieval, catastrophic R². The prefix LOFO ridge and the n=50 context ridge sit bottom-right: positive R², chance retrieval. Fitted 963k maps hold the top-right corner. All values come from the banked on-policy capture; dashed line = context-arm chance 0.001.

Figure. The full ladder-by-metric grid behind the scatter. Context arm (left; 963k-train rungs, pool 1,000) and prefix arm (right; leave-one-family-out, pool 50); brighter is better within a column. Median rank ties at its floor of 1 for every decent map at pool 1,000.

A single context-independent shift picks the right answer from 1,000 candidates half the time (0.532) while explaining less variance than the mean predictor (R² −0.920 vs −0.045); the constant mean sits at exact chance (0.001) with the least-bad R². The orderings invert; neither metric subsumes the other.

The nonlinear edge over ridge grows with candidate-pool size

acc@1 (euclidean, n=1000) against candidate-pool size per 963k-train estimator on the left; on the right, paired nonlinear-over-ridge acc@1 gaps with 95% bootstrap whiskers on shared draws. Companion: each true answer's rank among 100,000 candidates.

Figure. Accuracy decays with pool size; the nonlinear maps decay slowest. The width-8192 gap over ridge grows 0.036, 0.061, 0.079, 0.101 across pools 1,000 to 100,000; the width-32768 map reaches +0.115 and the kernel map +0.078 at 100,000. Dashed line = chance 1/pool.

Figure. Per-row rank distributions at the 100,000 pool, one step curve per estimator. Fitted maps place most true answers at rank 1; identity plus bias needs ranks in the hundreds for its tail; the constant mean concentrates near rank 50,000, the random-guess region.

The nonlinear R² gain (+0.056 for width 8192) buys discriminability that compounds with pool size: the gap growth (+0.065) excludes zero on shared draws and holds within each pool-composition class, so pool size rather than composition drives it. The pool-1,000 gap is 3-4 times larger at layers 14 and 26 (+0.135 and +0.123 against +0.036), where ridge is weaker.

Every shuffled-pair null collapses to its floor, and raw cosine's floor is high

Per metric panel: observed value per estimator (dot) against its 200-draw shuffled-pair null (violin); retrieval panel at the 1,000-candidate test pool, n=1000 rows.

Figure. Nulls collapse to metric-specific floors. Retrieval nulls sit at analytic chance (mean 0.00092 vs 0.001); R² nulls are strongly negative (ridge −0.76); the cosine null is 0.68 for ridge predictions while the constant train-mean scores 0.798.

No metric shows a non-collapsing null on the fitted maps' pipeline — the instrument reads as sound. Cosine is the one anticipated exception: a mismatched prediction-target pair still scores 0.68 because activation space shares a dominant mean direction, mean-cosine values above roughly 0.7 carry almost no information about map quality. Never headline a map on cosine without this floor; the WildChat round reproduces the collapse on its own battery (ridge R² null −0.75, cosine null 0.71).

Retrieval hubs are corpus-structured; the CSLS correction helps every estimator

The left panel shows the corpus composition of each estimator's top-100 cosine hubs at the 100,000 pool, with the pool-composition reference. The right panel plots paired CSLS-minus-cosine acc@1 gains per estimator and pool (95% bootstrap whiskers, n=1000).

Figure. Hubs sit where each estimator's predictions land. Fitted maps' hubs are test-row-dominated (61-70 of 100 against 1% of pool); identity-family hubs are train-region LMSYS distractors (87 of 100). Fitted-map CSLS gains span +0.027 (width-32768 map, pool 20,000) to +0.097 (ridge, pool 100,000); identity copy gains up to +0.195.

Cosine 10-occurrence skewness reaches 13.6-25.8 against a Gaussian reference of 3.2; the euclidean excess is modest (23.8 vs its own Gaussian reference of 20.8) — mostly a cosine phenomenon. Hubs follow each estimator's prediction region, so part of the CSLS gain compensates corpus and density structure beyond generic hubness; it helps weak estimators most (identity copy +0.195 at pool 1,000) — so plain kNN understates weak baselines.

At small training sets the retrieval ranking flips while pooled R² fails the other way

Ridge versus identity-plus-bias acc@1 across four training regimes (963,444 / 3,600 / 50 context rows; prefix leave-one-family-out ≈43; euclidean; chance dashed per regime), paired with the per-family prefix ridge R² behind the pooled value.

Figure. Ridge's discriminability is bought with training data; the bias estimate is not. Ridge falls 0.805, 0.720, 0.065, 0.04 across the regimes while identity plus bias holds 0.532, 0.503, 0.501, 0.84. All seven prefix families sit at or below zero R² against a pooled +0.54; prefix R² is estimator-degenerate (≈43 rows, 3,584 dimensions), never compared to context-arm values.

The prefix arm reproduces the previously measured dissociation (identity-plus-bias acc@1 0.84 vs ridge 0.04) to the digit; the n=50 context rung reproduces the flip with the data, pool, and folds all held fixed: a small-training-set property (prefix-specific contributions bounded, not excluded). The mirror failure: shrunk predictions near the fold mean track family membership with no per-context discriminability; the n=50 context rung shows an attenuated version (R² +0.384, acc@1 0.065 vs chance 0.001). The R² ordering never flips.

Every cross-class estimator ordering transfers to fresh held-out WildChat targets

Side-by-side estimator-by-metric heatmaps: the banked LMSYS test-1000 context arm (left) against 1,000 fresh held-out WildChat targets (right); 14 arms by 6 metrics, layer 19, targets-only pool, each arm scored against its own corpus's targets. Companion: per-arm paired values for euclidean acc@1 and pooled R².

Figure. The ladder-by-metric pattern is corpus-stable. Left: banked LMSYS arm; right: WildChat held-out targets (n=1000, fingerprint-screened out of the training pool). Rank agreement per plotted metric: euclidean acc@1 τ 1.00 with zero inversions; mean cosine 0.98; CSLS acc@1 0.98; mean reciprocal rank 0.96; pooled R² 0.93; median rank 0.92.

Figure. Per-arm values barely move between corpora at the targets-only pool. Paired LMSYS and WildChat points per estimator: euclidean acc@1 (left panel) and pooled R² clipped at −2 (right panel). The largest shift is identity copy, 0.254 to 0.215. At the 100,000 pool WildChat targets retrieve better: ridge 0.630 vs 0.544, wide map 0.736 vs 0.659.

With no refit and the same pinned weights, every parent-separated cross-class ordering reproduces on WildChat: fitted maps above identity plus bias above copy above the chance-level mean on retrieval, positive fitted R² against negative baselines, and both dissociation directions (identity plus bias 0.527 acc@1 at R² −0.876). The wide map's retrieval edge over ridge again grows with pool size, +0.044 to +0.106.

The two resolved R² flips sit inside the identity-plus-bias family — the mixed-corpus bias beats its LMSYS-fit companions on WildChat and trails them on LMSYS, at most 0.06 R² units near −0.9 — mean-alignment, not a class reordering. The maps trained on 434k WildChat rows: held-out-row transfer, not corpus-OOD.

Memorization inflates retrieval far more than variance explained

Paired in-train vs held-out dumbbells per estimator: euclidean acc@1 at the 1,000-candidate targets-only pool (left panel) and pooled R² clipped at −2 (right panel); 1,000 deliberately in-train WildChat rows — a contamination-labeled memorization read — against the round's 1,000 held-out targets; no refit, layer 19.

Figure. Retrieval moves, variance explained barely does. Fitted 963k maps gain +0.074 to +0.109 acc@1 on in-train rows (wide neural map 0.947 vs 0.844; ridge 0.874 vs 0.800) while their pooled R² shifts by at most 0.03; the identity-plus-bias and copy baselines gain +0.026 to +0.042; the constant mean stays at chance.

On rows the maps trained on, fitted-map retrieval inflates by +0.074 (ridge) to +0.109 (small neural map), while the fitted maps' pooled R² moves at most 0.03 in either direction — ridge and the kernel map even read slightly lower in-train. The identity-plus-bias and copy baselines, whose global shift cannot memorize rows, gain only +0.026 to +0.042 on the same target swap, bounding the row-difficulty component; roughly +0.04 to +0.07 of the fitted-map gain is memorization proper.

Retrieval rewards per-row placement, which training retains; variance explained aggregates over dimensions and barely separates seen from unseen rows. In-train reads of fitted maps overstate deployable retrieval, not R².

The generic boundary control stays at least 0.29 R² below the chat map from 1,000 training pairs up, and no robustness check closes the gap

The paper's scaling figure: held-out R² and retrieval acc@1 against training-set size, banked chat curves beside the WikiText "." control ridge. Error bars show across-draw spread through n=2,500; above, control bars are article-bootstrap intervals while chat bars begin at 50,000 (its ≤25,000 cells are banked-point-degenerate); the binding 963,444 gap carries intervals on both sides.

Figure. The control scales but never approaches the chat map. The generic boundary token ('.') curve rises from R² 0.040 to 0.440 and from acc@1 0.013 to 0.344 across the grid, while the chat ridge runs 0.32 to 0.754 and reaches acc@1 0.805 at the top rung. All 15 banked chat training sizes are covered (top common rung 963,444).

Figure. The per-cell companion to the scaling summary: no layer rescues the control, and the draws barely scatter. Held-out R² per layer (2, 14, 19, 26); layer 26 reads 0.448 at 963,444 pairs against layer 19's 0.440, layer 14 reads 0.356, layer 2 reads 0.235; the layer-19 dots show all three draws per small rung spanning at most 0.05.

From 1,000 pairs up, the gap's 95% lower bound stays above 0.29 R² (minimum 0.2925 at the two-sided top rung), meeting the plan's control-distinct rule (Methodology). No tested layer closes the gap: layer 26's best 0.448 sits about 0.31 below the chat map, layer choice moving R² up to 0.205. Type pooling shifts R² by 0.003, a 6-per-article draw cap by 0.0001; shuffled-pair nulls collapse.

The "!" and "?" arms are N/A — not tested (14 and 9 eval spans). The open pilot-resume-flat-basis concern is rerun-scheduling-only (a resume before the first gate must redo warmup and probe, not gate on the flat shard projection); it alters no reported value.

The neural map keeps gaining past the linear plateau on a single within-store training-size ladder

Held-out pooled R² against training-set size for paired ridge and width-8192 neural-map fits at 8 fresh rungs (5,000-500,000 rows) from the 963k-context store, scored on one pinned 1,000-row test pool; open squares: banked small-n anchors; diamonds: the banked 963,444 apply; right panel: whitened-CSLS acc@1 under the 0.94 duplicate ceiling. A companion shows per-rung deltas and the verdict slope gap against its margin.

Figure. The neural map keeps gaining where ridge plateaus. Within one store and one eval pool, ridge rises 0.744 to 0.761 over 50,000-500,000 rows while the width-8192 map rises 0.761 to 0.807; open squares are banked small-n anchors from the other store, diamonds the banked 963,444 apply; identity plus bias sits near −0.90.

Figure. Two panels: per-rung paired deltas cross zero near 25,000, and the verdict slope gap clears its margin. Left: paired neural-minus-ridge R² deltas on 1,000 shared context-bootstrap draws: negative at 5,000 and 10,000, positive from 25,000 on, +0.046 at 500,000; every rung's 95% interval excludes zero. Right: the 50,000-to-500,000 slope-gap bootstrap interval against the 0.01 verdict margin.

Ridge leads at the smallest rungs (0.689 against 0.669 at 5,000), the arms cross near 25,000, and the neural map at 50,000 rows already matches ridge at 500,000 (0.761 both). Past 50,000 ridge gains 0.016 while the map gains 0.047; the slope gap's 95% lower bound is +0.028 against the 0.01 margin, and every paired bootstrap draw clears it — Confirmed. Whitened-CSLS acc@1 barely separates the arms near the 0.94 duplicate ceiling (0.898 against 0.896 at 500,000) while euclidean keeps the edge (0.844 against 0.811) — the parent's metric dissociation on a new axis.

The published cross-store scaling join compared curves on different eval pools and is superseded

Held-out R² per training size as two store-faithful curves — the earlier scaling store's cells at 25,000 and below on their own pinned eval rows, beside the fresh within-store rungs — with the shared-size value gaps marked.

Figure. Same nominal training size, two different evaluation pools. The fresh within-store rungs (filled) read 0.003 to 0.009 R² above the earlier store's cells (open) at the shared 5,000-25,000 sizes; the two stores' pinned eval pools share 0 of 400 checked rows (median nearest-neighbor distance 6.63).

The round's planned identity gate measured the mismatch directly: none of the earlier store's 400 pinned validation rows content-matches any row of the 963k store's pinned bundle, so the published joined ladder — one axis mixing the two stores' cells — compared curves scored on different held-out rows. The shared-size gaps are small but comparable to per-rung slope increments, and they reflect the whole store difference (draw scheme, capture batch, eval pool), not one attributable knob.

The within-store ladder replaces the join; a machine-readable supersession record routes the joined figure's consumers, and the paper and poster re-render is filed. The boundary-control gap classification is unaffected: it compares chat and control values per rung, never the chat curve's own slope.

---

Rollout averaging primarily improves variance explained

With maps frozen, K=1→5 raises linear/nonlinear R² from 0.751/0.804 to 0.803/0.859 and corrected top-1 from 96.37%/96.67% to 97.35%/97.88%. The original curve averages all 31 subsets; retrieval is not strictly monotonic.

K=10 raises R² to 0.8103/0.8652: paired gains +0.00768 [0.00606, 0.00925] and +0.00642 [0.00448, 0.00825]. Retrieval reaches 97.88%/98.09%: +0.53 [0.11, 1.06] and +0.21 [−0.21, 0.64] percentage points; the nonlinear improvement remains unresolved. Identity plus bias moves from R² −0.9793 to −0.9904 and retrieval 71.13% to 72.93%.

Figure. Ten draws yield small additional gains. R² uses 1,000 held-out contexts; whitened-cosine/CSLS retrieval uses 942 fixed identities (chance 0.106%). Bars are pointwise 95% intervals; paired contrasts use 2,000 shared bootstraps, conditional on the observed banks, frozen maps and fixed pool. Exact results and validation.

New-five-only R² is 0.8036/0.8563; differences from the old five have intervals including zero, which does not establish equivalence.

All 5,000 new vectors and planned cells completed; parity and independent review passed. Raw data and paired evidence are archived. Training-K effects and optimal K remain untested.

Repro: Parent run 0 GPU-h — VM CPU only, ~71 min wall (context battery 1,884 s; distractor stream 448 s; prefix battery 1,886 s; 19.7 GB peak RSS, above the plan's 12 GB projection and its 16 GB reroute line, and the launch-time OOM-priority protection silently failed via a pid-capture artifact — a rerun should route to a cpu-mid instance). Battery driver: scripts/issue1901_metric_battery.py, run at worktree commit 680f322678 (JSON-metadata code SHA c63eca8d1e; characterization amended at d36c7c5dc5); body figures regenerated by scripts/issue1901_body_figures.py at commit 76a1a9826d. Eval JSONs: metric_battery tree (context_arm.json, prefix_arm.json, metric_characterization.json, distractor_manifest.json, boot_draws_context.json, boot_draws_prefix.json). Figures + sidecars: figures/issue_1901 tree — the tree's pre-v2 hero_ladder_by_metric.png is superseded by the v2 scatter and grid pair, not embedded. Distractor pool npz: HF issue1901_metrics/analysis_tensors. Reused artifacts: Reused weight payloads from #779: issue779_monitoring/n1m_readout/weights — fit: the exact banked mixed-corpus fits; applied-vs-banked ridge R² delta 8.1e-11, all 12 payload key checks PASS. Reused context/answer capture bundle from #779 (pass_b/train_context_vectors.pt, byte-size-asserted 6,021,122,751) plus the seeded distractor stream from its capture chunks — fit: same pinned split, same layers. Reused prefix battery stores from #722 (eval_results/issue_722/identity_bias_knn + the c_C/v_A stores its loaders resolve) — fit: verbatim leave-one-family-out fold structure; reproduces the banked read exactly. WildChat follow-up round (wildchat-target-battery, source proposer-9b-cheap, round 1): VM CPU phases (candidate screen; battery wall 1,681 s at 20.7 GB peak RSS — again above the plan's 16 GB comfort line, the same class as the parent's 19.7 GB note; figures) plus one fellows SLURM GPU capture phase (job 16114, 1 GPU, ~9 min wall against 1.5 h booked — roughly 0.15 GPU-h realized). The round's train-pool screen aborted its serial pilot at a projected 11,907 s under the plan's fail-loud budget gate; an 8-worker vectorized rewrite realized the screen in 222.7 s. Round code + results at commit 8cadf1e710 on origin/issue-1901-wildchat (battery run at worktree commit 011c7bef3c): wildchat_arm.json, transfer_comparison.json, boot_draws_wildchat.json, updated metric_characterization.json; round figures + sidecars in the round figures tree. HF round prefix issue1901_wildchat: manifest/ (pinned-revision stream + screen record), final_token_capture/ (3 activation chunks, ~112 MB), raw_completions/ (3 chunk files plus 1 over-length-skip sidecar; the round plan's section 6.5 labeled the raw-completion home final_token_capture/, the realized home is the Upload-Policy-canonical raw_completions/ — both populated, upload-verified). In-train memorization companion (the task's single zero-GPU free-analysis follow-up, screened not-redundant): VM CPU 3,396.8 s including a deterministic held-out re-run at identical values (companion battery wall 1,723.4 s, 20.9 GB peak RSS per its metadata — the fourth battery over the 16 GB comfort line; the cpu-mid rerun recommendation stands); results at commit 36e0b9ca19 (wildchat_intrain_companion.json, boot_draws_wildchat_intrain.json); companion figure + generator script at commit 0e7773eaa6. Boundary-token control round (generic-boundary-token-control, source user-chat, plan v10, run 2026-08-23): one 16-vCPU RunPod CPU pod leg (pair build + screens, ~40 min pod wall) and one 1× H100 RunPod pod leg (capture 643 shards, 39 fit units, publish; ~2.3 h wall ≈ 2.3 GPU-h realized against 5.5 booked); driver scripts/issue1901_boundary_token_control.py run at commit 789e64d265 (eval-JSON metadata SHA); eval JSONs committed at c43bf0c345 — boundary_token_scaling_L19.json, boundary_token_secondary.json — and figure + renderer + poster edits at 933b5419c3 (updated c1_scaling_train_pool + the exploratory dump). Inherited-figure regression gate PASS: all 304 previously committed scaling-figure points reproduce in the regenerated sidecar (tuple equality at 1e-9) beside the 120 new control points; rung-50 GPU-vs-CPU fit parity delta exactly 0. HF round prefix issue1901_boundary_ctl: manifest/ (54 files), capture/ (1,286 files: 643 bf16 tensor shards + per-shard row indexes), eval/ (2 JSONs) — upload-verified before pod teardown. Poster supersession: the MATS poster scaling plot previously drew a 0.1087 horizontal line labeled as the pooled control's R²; that constant is fraction_of_fulln_ceiling from the prior pooled-control artifact (the underlying transfer R² is 0.0732), and the plot now draws the measured single-type curve instead; the promoted-record correction is filed as #2495. One open code-review advisory on the round's driver (resume-window accuracy of the capture-pilot fence bookkeeping) touches ops tooling only, no result value. Dense-ladder round (mlp-scaling-densify, source user-chat, plan v15, run 2026-08-25): one 1× H100 RunPod pod (eval intent; realized ~1.5 h wall ≈ 1.5 GPU-h against 2.72 booked, including a smoke leg halted by the cross-store gate and one crash-fix leg), plus a VM-CPU fold analysis (bootstrap + figures; the 6.02 GB banked target bundle and the prediction files staged to the data disk under the user staging root as a re-downloadable cache). Driver scripts/issue1901_paper_densify_mlp.py run at commit 0477102ef1 (JSON-metadata SHA); results committed at 5a3df5a578 on origin/issue-1901-mlpdense — mlp_scaling_dense_L19.json and the machine-readable superseded_cross_store_join.json naming the joined figure's consumers; fold analysis + figures (issue1901_mlpdense_fold_analysis.py, issue1901_mlpdense_fold_figures.py, dgap_context_bootstrap.json) at commit 8666d15cba; round figures committed on main at 75988641cc (dense figure set). The cross-store gate measurement (0/400 content matches) is recorded in the task event log (2026-08-25T03:18Z) with the harvested smoke log committed at a22895e7b6. HF round prefix issue1901_mlpdense: analysis_tensors/ (23 files: 22 fp16 per-rung test-prediction npz + whitening stats) and smoke/analysis_tensors/ (9 files) — upload-verified before pod teardown. The paper and poster scaling-figure generators still draw the superseded cross-store join; the re-render is filed as #2570, and the claims ledger's training-size row is re-pointed at the within-store curve. Battery seed 1901 in all rounds; weight revision pin unchanged. Config slugs for the ladder rungs: const_mean, identity_copy, identity_bias, scaled_identity_3600, diagonal_only_3600, ridge, mlp_w8192, mlp_w32768, krr_nystrom, n3600_arm (const_mean_3600, identity_bias_3600, ridge_3600), ridge_n50_fixedlam, identity_bias_n50, prefix_arm, perm_null; WildChat round slugs: wc_heldout, repro_assert, wc_intrain (free-analysis companion); boundary round slugs: btok_dot, btok_ib, btok_mean, btok_null, btok_qm, btok_ex, btok_pooled, btok_grpdiv, btok_layers; densify round slugs: dense-ridge-n{n} and dense-mlp-n{n} over the 8 rungs (prediction files preds_L19_n{n}_{arm}.npz), selections lmsys_{n}, gates g1_cross_store (refuted by measurement), G2 refit parity, G3 pool floor.

Context: Originating prompt (verbatim, from task frontmatter):

Run this in background with happy coder:

Motivation

We have come up with a bunch of different metrics for measuring the quality of the mapping. I want to characterize each and see how the 1m context linear and nonlinear mapping on them vs baselines, as well as what each metric is measuring exactly

• [[I think a good baseline would be learned bias]], W = identity - another metric besides R^2 would be e.g. P(answer summary is in k nearest neighbors of prediction)

Follow-up round 1 wildchat-target-battery (source proposer-9b-cheap; run and folded 2026-07-31 UTC). Originating proposal (verbatim, from the follow-up scope marker):

WildChat-target battery (ladder ordering transfer to non-LMSYS targets) — Reproduction/OOD. [...] Hypothesis: the estimator-ladder orderings measured on LMSYS test targets (fitted >> identity+bias >> copy >> mean on retrieval; both dissociation directions) transfer to WildChat targets. Falsification: any ordering inversion on WildChat targets at matched pool sizes.

Follow-up round 2 generic-boundary-token-control (source user-chat; run and folded 2026-08-23 UTC). Originating ask (verbatim, from the follow-up scope marker):

Add a single generic-boundary-token control arm to the C1 scaling figure (paper outline [...]: "Scaling plot ... linear vs MLP vs identity + bias vs generic boundary tokens"; "NEW EXPERIMENT: look at mapping for single generic boundary token, we have it averaged across boundary tokens"). The existing control [...] averages across boundary-token types; this arm fixes ONE generic boundary token type and fits the map from its residual state to the next-span/whole-answer mean at the same training sizes, layers, and eval pools as the headline scaling ladder, then adds the curve (held-out R2 AND acc@1) to figures/paper/c1_scaling_train_pool.pdf alongside linear / MLP / identity+bias.

Follow-up round 3 mlp-scaling-densify (source user-chat; run and folded 2026-08-25 UTC). Originating ask (verbatim, from the follow-up scope marker):

are you sure we don't have more data. If it's really missing then rerun it

— posted against the paper-plan claim "The linear mapping's performance plateaus at around 50k contexts, with nonlinear seeing slight gains past that"; the scope marker verified the nonlinear scaling artifacts missing above 10,000 training rows before the round trained the fresh rungs.

Lineage: fresh direction (no parent task); characterizes the metrics used across the #779 / #722 mapping line. Created 2026-07-30; parent run 2026-07-31 (UTC); interpretation settled 2026-07-31 after one revision round; WildChat transfer round folded 2026-07-31; generic boundary-token control round folded 2026-08-23; dense-ladder round folded 2026-08-25.

Figure access: The historical K=1–5 curve is deliberately linked; the K=10 comparison is embedded.