The context→answer map's error is category-structured: non-English contexts are predicted better than English, and the linear-vs-nonlinear gap concentrates in low-variance answer directions (HIGH confidence)

Methodology: docs/methodology/issue_1482.md · gist

Takeaways

• Non-English contexts are predicted better than English (normalized error 0.274 vs 0.291), falsifying the planned direction; the sign survives intrusion exclusion and corpus transfer, and floor adjustment widens the gap to −0.034 (CI95 −0.044 to −0.024).
• Structure inside the arms dwarfs the binary gap: per-language means span German 0.236 to Arabic 0.420; task types span small talk 0.235 to translation 0.337 (all 15 contrasts pass BH-FDR).
• The nonlinear map wins on 90.7% of contexts, and its advantage grows about fourfold toward low-variance answer directions (0.035 to 0.138), surviving per-direction ridge tuning: genuine nonlinearity, not shrinkage.
• Feature predictability is an answer-side property (Spearman 0.93); within-answer consistency dominates at every depth and dictionary (ρ 0.60/0.68/0.60 at layers 19/3/20), with no early-layer inversion (median 0.095 layer 3 vs 0.129 layer 19).
• Training-imposed matryoshka tiers give a judge-free granularity read: median R² falls 0.43 → 0.17 → 0.04 general-to-specific, the gradient survives activity stratification (Spearman −0.39 vs band −0.25 to −0.23) but reverses in the two lowest-activity quintiles; judged labelling stayed frozen.
• The pile-trained twin reconstructs chat worse (variance explained 0.71 vs 0.80) yet reads slightly more predictable (median difference 0.017, unpaired panels); coarse features are shared across training corpora (tier-0 match cosine 0.57 vs 0.08 at tier 2).

Goal

This experiment in context: The parent #779 fit five predictors from the last-prompt-token activation c_last(x) to the mean-answer activation v(x) (layer 19, Qwen-2.5-7B-Instruct) on ~963k real LMSYS/WildChat chat contexts and measured what the map achieves overall: held-out R² 0.754 (ridge) to 0.810–0.813 (MLPs). This experiment asks where the remaining error and the nonlinear-only component live — which contexts, which answer-side SAE features, which directions of the answer representation. The refits here reproduce the parent's committed aggregates on the same pinned split with the same code (directly comparable by construction), and the teacher-forced capture recipe with its token-id-concatenation seam rule and identity gate is inherited from #1092.

Broader narrative: the project treats the pre-generation context state as a predictor of what the model is about to do, and the leakage-theory line assumes that context→answer map is linear. Whether the map's error is category-structured (an actionable trust boundary plus a fix path) or diffuse (an intrinsic answer-variability ceiling) decides how far a pre-generation behavior predictor built on this map can be trusted; the per-direction linear-vs-nonlinear decomposition tests what the linear assumption discards.

Methodology

Design: An error-analysis of an existing map — no new training. One GPU run (three GCP 4×A100 attempts, then a RunPod 4×H100 failover pod; checkpointed phases seeded from the surviving partial attempt), then off-pod judging and analysis. Stages: (1) stream the parent's 1,920 capture chunks from HF and rebuild the 963,444-context train matrices (layer-19 last-prompt-token state and mean-answer state), re-asserting the parent's pinned 400-row val / 1,000-row test split by sha; (2) carve a fresh 20,000-context holdout (RNG seed 1482, corpus-stratified, never entering any fit) plus a disjoint 120,000-context SAE fit subsample; (3) refit all five parent fitters three ways — full-train (the reconciliation control), holdout-excluded (the taxonomy read), and LMSYS-only (the corpus-transfer fold) — persisting per-context error; (4) teacher-forced forwards over the parent's persisted prompt+answer text (per-segment token-id concatenation, never re-tokenizing the concatenated string) for the 141,400 SAE-arm contexts, SAE-encoding every token and pooling answer-side features three ways (mean, max, fraction-active); (5) linear and MLP maps in feature space — context features → answer features, dense context state → answer features, and a prefix-arm null — plus a per-direction PCA decomposition of the dense map; (6) one judge call per context labeling language, topic, refusal-adjacency, answer-is-refusal, and format; (7) vectorized bootstrap/permutation analysis and figures. The dense map is the context-based arm (prefix + user query up to the last prompt token). A zero-GPU follow-up round (2026-07-19) added stage (8): a seeded label-permutation null for the per-feature read — answer rows permuted within the 120,000-row SAE fit set, λ pinned to the parent-selected value, feature restriction asserted equal to the parent's feature ids, holdout scored on true pairs; an identity-permutation gate reproduced the parent's committed per-feature R² (median |Δ| = 1.5e-5; pooled 0.6901407 vs 0.6901409). A second follow-up round (k-resample-noise-floor, 2026-07-19, ~3.1 realized GPU-h) added stage (9): regenerate K = 4 fresh answers per context (seeds 43–46, the parent decoding recipe) for a 2,000-context language-stratified subsample of the labeled holdout (1,000 per arm, RNG seed 148201), capture each fresh answer's v under the parent capture convention (full-template re-tokenization, answer span including the end-of-turn tail), stream the parent seed-42 v verbatim from the parent capture chunks (2,000 of 2,000 rows found across 1,233 chunks; no recapture), and compute the per-context answer-sampling floor plus the floor-adjusted language contrast (joint 10,000-draw bootstrap, seed 1482). Two earlier attempts of this round recaptured the parent draw under the generation-time token-id convention and failed the identity gate — generation-time ids differ from the parent's own full-template re-tokenization for 93.1% of rows — so the round switched to the streamed-parent + parent-convention design before any read. A third follow-up round (perdirection-shrinkage-control, 2026-07-19, ~1.3 realized GPU-h) added stage (10): rebuild the same train matrices, pinned split, and top-256 answer-PCA basis (sha- and value-gated against the committed round), replace the linear baseline's single global λ with a per-direction λ selected on the pinned 400-row val over a 38-value grid, score the tuned map on the same 20,000-context holdout through the identical fp16-cast prediction convention with the MLP arm untouched, and re-read the band-gap excess with a 10,000-draw paired context bootstrap (seed 148202). A fourth follow-up round (feature-correlates, 2026-07-27, 0 GPU-h, user-chat inline) added stage (11): two per-feature covariate reads over the committed per-feature R² — within-answer consistency (mean ans_frac, the fraction of the answer's tokens where the feature is active conditional on being answer-active, accumulated over the 120,000 fit-tagged rows from the pooled store; the recomputed activity covariate matched the committed one exactly, max |Δ| = 0) and a judged abstraction level (300 features stratified by activity decile × within-decile R² tercile, seed 14822026; project judge over each feature's top-8 firing answers plus the Neuronpedia 19-resid-post-aa/trainer_1 auto-interp description as labeled auxiliary evidence — index-aligned to the run's k=64 trainer; reason-then-label, max_tokens 400, 60-item test-retest). A fifth follow-up round (early-layer-arm, 2026-07-29, ~4.6 realized GPU-h of 4 approved) added stage (12): an early-layer token-level arm — teacher-forced forwards over a corpus-stratified 30,000-context subsample of the parent corpus (24,000 fit rows drawn inside the parent SAE-fit set, 6,000 score rows drawn inside the parent holdout; RNG seed 14823; parent split shas re-asserted), capturing layer 3 alongside a fresh layer-19 control per token, encoding with the public layer-3 dictionaries (k=64 primary, k=128 replication), pooling answer-side features per the parent recipe, and refitting the context-features → answer-features and dense-input maps at both depths on the same rows with the same shared-Gram ridge (plus the parent MLP recipe on the layer-3 dense arm), a K = 20 label-shuffle null, and a per-feature covariate battery (activity, within-answer consistency, direct-logit footprint, co-activation, dense-latent flag, decoder–persona-direction alignment). Judged feature-labelling was frozen this round by user directive (plan v17 §0.-1; the prior labelling instrument was retracted as unreliable): the two planned judged-label permutation tests emitted deferred stanzas, a 377-feature mechanical evidence packet was persisted for the future labelling resume, and no feature-level abstraction or persona claim is made from this round. All planned fit units realized, including the layer-3 dense-input MLP cell missing from the round's first leg (a same-day instance restart superseded that leg; the final leg re-ran capture through upload end to end). A sixth follow-up round (matryoshka-tier, 2026-07-30, ~5.6 realized GPU-h of 6 approved) added stage (13): the early-layer round's exact 30,000 rows (split shas re-asserted) captured at layer 20 and encoded through two matched matryoshka dictionaries — chat-trained and pile-trained twins differing only in training corpus (jumprelu-family batchtopk, k = 100, nested widths 2,048 / 16,384 / 65,536, so the feature id determines a training-imposed granularity tier 0/1/2) — pooling answer-side features per the parent recipe plus a sink-robustness twin; per dictionary, the context-features → answer-features shared-Gram ridge on a 16,384-feature tier-stratified random panel above the activity floor (a stated deviation from the parent's top-activity panel: top-activity selection would distort the tier × activity joint distribution the headline conditions on and could censor tier 2), a dense-input companion, a prefix-arm null, a K = 20 label-shuffle null, the covariate battery, identity-plus-bias and retrieval baselines, cross-dictionary decoder matching, and a within-activity-stratum tier-label permutation as the headline test. Judged feature-labelling stayed frozen this round too (zero judge calls): the tier × judged-level scope item is deferred to the successor labelling instrument, with 360 + 180 label-free evidence packets persisted for that resume.

Conditions: the full-train reconciliation refit; the holdout-excluded refit (plus a second MLP seed); the LMSYS-only transfer refit; the feature-space map from context features; the dense-input feature map; the prefix-arm null; the encode-the-prediction read; the SAE fitness pilot; the split-half stability read; the per-feature shuffle null; the k-resample floor arms (English and non-English); the k-resample MLP and raw-units re-reads; the per-direction shrinkage-control arms (val-tuned and holdout-oracle ridge); and the early-layer arms — the layer-3 context-features map (mean/max/fraction-active poolings), its sink-robustness pooling twin and k=128 second-dictionary twin, the layer-3 dense-input map plus its MLP twin, the layer-3 prefix-arm null, and the matched-row layer-19 context-features and dense-input controls. The matryoshka-tier round adds: the chat-dictionary and pile-dictionary feature maps (mean/max/fraction-active poolings), the sink-masked pooling twin, the dense-input companions, the prefix-arm nulls, the label-shuffle nulls, the within-stratum tier permutation, the identity-plus-retrieval baseline pair, and the cross-dictionary matching read. (The condition-name → artifact-filename legend is in the footer.)

Scope caveats:

• The SAE arm covers new-pool rows only — the 1,400 pinned val/test contexts have no reconstructable text, so feature-space λ-selection uses a 2,000-row carve of the SAE subsample.
• The Goal-named 16,384-feature SAE repos are weight-empty on HF, so the run substitutes the only weight-bearing suite (andyrdt layer-19 instruct SAEs, 131,072 features) — better matched to the map layer, validated by the fitness gate below.
• The dense per-context read is context-arm-only: every prompt is single-turn, so the prefix (everything before the user query) is one constant chat-template string — verified live (prefix-end states constant across contexts, min cosine 1.000) — and the prefix arm runs at SAE-subsample scale as a null.
• The encode-the-prediction read is off-distribution (an SAE applied to mean-pooled states) and reported as secondary only.
• The conversation-depth taxonomy axis was dropped (single-turn corpus).
• 4 content + 29 transport judge losses (0.165%) were not re-judged — an adversarial bound assigning all 33 missing rows extreme observed values moves the language contrast by at most +0.011 (to −0.006), so its sign cannot flip.
• v(x) is one sampled answer per context, so absolute per-category error levels include an answer-sampling-variance share — measured by the k-resample round at 33–40% of per-context error on the language arms (the paired gap reads are immune — same answer under both fitters).
• The early-layer arm's judged labels are frozen: the planned pooled-depth and within-layer-3 abstraction-level permutation tests are deferred until a validated auto-interp labelling instrument lands (the successor labelling task named in the Context row), so this round carries mechanical reads only.
• The early-layer arm fits 24,000 rows (vs the parent SAE arm's 120,000), so cross-round absolute levels are not comparable; the matched-row layer-19 refits carry the depth contrast. The cross-depth absolute comparison is itself descriptive — the two dictionaries differ in reconstruction quality (measured fraction of variance explained 0.920 at layer 3 vs 0.810 at layer 19).
• The layer-3 MLP read is fit on 24,000 rows (the committed dense MLP fit 963,444), so its deficit against the ridge conflates depth with fit size; all early-layer fits are single-seed (fit seed 0, subsample seed 14823), with the k=128 second dictionary as the replication axis.
• The co-activation covariate saturated at 1.0 for over 99.9% of features at both depths (full-density companion latents), so its near-zero rank correlation is uninformative, not evidence of no effect.
• The matryoshka round's layer-20 jumprelu-matryoshka k=100 numbers are not directly comparable to the committed layer-19 BatchTopK k=64 numbers: the core battery (fits, nulls, baselines, covariates) is recomputed on this round's own arms, no figure splices tier onto committed values, and cross-round correlate comparisons are qualitative only.
• Tier is confounded with firing frequency by construction (general-tier features fire more often), so the raw tier gradient is descriptive; the headline is the within-activity-stratum permutation read, with activity deciles merged to quintiles per the plan's fallback (the sparsest decile-tier cell held 17 features, under the 20-feature floor).
• The matryoshka fits use 24,000 rows and a single fit seed per arm; the tier-stratified target panel keeps every floor-clearing tier-0 feature (1,640 chat / 1,639 pile) but samples tiers 1–2 (6,144 and ~8,600 of the floor-clearing populations), so panel composition affects which features carry R² estimates, not the full-population descriptives. The chat-vs-pile predictability comparison is additionally unpaired over these differently-selected populations: each panel's activity floor is computed on chat tokens, so the pile panel conditions on chat-transferability.
• The tier × judged-abstraction-level agreement item from the round's scope is deferred: judged labelling stayed frozen (zero judge calls), and the judged spec is preserved verbatim in the evidence pack for the successor instrument's resume.

Training: N/A — no model training (frozen Qwen-2.5-7B-Instruct forwards only). The inherited fit and instrument hyperparameters:

Hyperparameter	Value	Source
Fit seed (all fitters)	0 (the parent's; the generation engine seed was 42 — distinct)	parent fit record eval_results/issue_779/fitter-fair-comparison-n1m/n1m_fits.json
Ridge λ grid	logspace(−3, 8, 23), selected on the pinned 400-row val	parent fitter scripts/issue779_ffc_n1m_fits.py at bd9f6865de16
MLP fitters	widths 8,192 and 32,768; lr 3e-4; batch 4,096; internal-val early stop; residual-skip variant	same
KRR-Nyström centers	16,384, passed explicitly (the script CLI default is 8,192)	parent fit record krr_grid.nystrom_centers
Train / eval sets	963,444 contexts (LMSYS 529,085 / WildChat 434,359); pinned val 400 / test 1,000 (sha-asserted); fresh holdout 20,000 (seed 1482)	parent fit record + split_1482.json
SAE	andyrdt/saes-qwen2.5-7b-instruct revision c37e53c4bb, resid_post layer 19, BatchTopK, 131,072 features; k=64 primary, k=128 robustness	the suite's config.json + eval_results.json; arXiv 2412.06410
Feature restriction	16,384 answer-side + 8,192 context-side most-active features (floor: active in ≥1,200 of the 120,000 fit rows)	realized sae_perfeature/summary.json
Shuffle-null draws (follow-up round)	K = 20, seeds 1482000–1482019; λ pinned to the selected 10,000	sae_perfeature/shuffle_null.json recipe block
Judge	claude-sonnet-4-5-20250929, Batch API, reason-then-label, max_tokens 400, temperature 0, 1 draw + 200-item test-retest	project judge pin; judge_labels/
Statistics	10,000-draw bootstrap CIs; 10,000-draw permutation p + BH-FDR q=0.05 over the 15 exploratory contrasts	taxonomy.json, h1_contrast.json
K-resample draws (follow-up round)	K = 4 fresh answers per context, per-request seeds 43–46; engine seed 42; temp 1.0, top_p 0.95, max 1,024 new tokens (inherited); parent seed-42 draw streamed, never regenerated	round plan §11 dof arithmetic; #779 generate script issue779_ffc_n10k_generate_capture.py
K-resample subsample	2,000 contexts (1,000/arm), language-code stratified within the non-English arm, RNG seed 148201	kresample/strata.json; round plan §11 power arithmetic
Floor estimator	ddof-1 trace of the within-context covariance of v over the four fresh draws ÷ the parent's stored per-context denominator; map share = squared prediction error to the fresh-draw mean minus trvar/4; the 5-draw variant reported as shadow (floor difference 0.0175 vs 0.0166)	driver scripts/issue1482_kresample.py; kresample/gates.json
K-resample bootstrap	10,000 joint draws, RNG seed 1482, resampling holdout rows and subsample rows within arms per draw	kresample/adjusted_contrast.json
Per-direction λ grid (shrinkage-control round)	38 realized values: 1e-9 (effectively unregularized) ∪ logspace(−5, −2, 16) ∪ the parent logspace(−3, 8, 23); parent λ = 0.001 at grid index 11; per-direction argmax of val R² on the pinned 400-row val, fp64 selection	round plan §11; parent selection convention scripts/issue779_ffc_n1m_fits.py; realized pdshrink_summary.json
Shrinkage-control bootstrap	10,000 paired draws over the 20k holdout contexts, RNG seed 148202, both arms recomputed per draw, direction means frozen at full-holdout values	perdirection-shrinkage-control/pdshrink_summary.json
Early-layer SAE (follow-up round)	same suite, resid_post_layer_3: k=64 (trainer_1) primary — fraction of variance explained 0.9204, L0 67.6 on 32,040 pilot tokens of this run's own activations (published 0.9309); k=128 (trainer_2) replication — 0.933 (published 0.9421); layers 1 and 5 rejected (0.7145 / −1.33)	early_layer/early_pilot.json; the suite's per-layer configs
Early-layer subsample	30,000 contexts: 24,000 fit (13,180 LMSYS / 10,820 WildChat) + 6,000 score (3,295 / 2,705), corpus-stratified at the parent LMSYS fraction 0.549, RNG seed 14823; fit ⊂ parent SAE-fit set, score ⊂ parent holdout, disjointness asserted in-driver	early_layer/split_early.json
Early-layer fits	shared-Gram ridge, λ selected on a 2,000-row carve of the fit rows (realized per-arm λ 3.16e3–3.16e5); feature restriction 16,384 answer-side / 8,192 context-side (activity floor 240 of 24,000, the parent's 1% recipe); MLP width 8,192, lr 3e-4, fit seed 0, 280 epochs realized	round plan §4/§11; early_layer/early_summary.json
Early-layer nulls, masks + covariates	label-shuffle K = 20 (λ pinned to the selected value); permutation/bootstrap seed 148230, 10,000 draws each; reference token mask = BOS offset 8 + 10× norm-outlier exclusion, sink-robustness mask adds special-token positions; covariates: activity, within-answer consistency, direct-logit footprint (top-20 concentration), co-activation, dense flag (activity > 0.5), decoder–persona-direction max-cosine (raw + population-centered); mean-to-spread ratio γ reported per depth (0.557 layer 3 / 0.664 layer 19)	early_layer/mech_tests.json; early_layer/early_pilot.json
Matryoshka dictionaries (follow-up round)	chanind/qwen2.5-7B-it-layer-20-saes revision db8c88b400e36e603d0563abcf8d289e5eff5687, hook layer 20, d_in 3,584, d_sae 65,536, k = 100, matryoshka widths 2,048 / 16,384 / 65,536 → tier = feature-id bin; chat-trained (lmsys) + pile-trained twins, training corpus the only difference	the repo's cfg + runner_cfg at the pinned revision; arXiv 2503.17547
Matryoshka encode semantics	SAELens v6.37.6 jumprelu verbatim: subtract b_dec, project through W_enc + b_enc, ReLU gated at threshold; decode through W_dec + b_dec; fp32; decoder-norm rescaling folded at conversion; the round-trip fraction of variance explained is the end-to-end assert	pinned SAELens tag source; matryoshka_tier/m_pilot.json
Matryoshka subsample + split	the early-layer round's exact 30,000 rows (24,000 fit / 6,000 score, subsample seed 14823; fit/score shas re-asserted); λ selected on a seeded 2,000-row carve (22,000 train / 2,000 val / 6,000 test)	matryoshka_tier/m_split.json; m_summary.json
Matryoshka fits	shared-Gram ridge over the parent λ grid (selected λ 1e4 chat mean-pooling; 3.16e4 pile and the max / fraction-active poolings; 3.16e3 dense-input); input restriction top-8,192 by activity; target panel 16,384 tier-stratified random above the 240-row floor, panel seed 14824 (all floor-clearing tier 0 — 1,640 chat / 1,639 pile — plus 6,144 tier 1 and ~8,600 tier 2)	matryoshka_tier/m_summary.json
Matryoshka nulls, tests + covariates	label-shuffle K = 20 (seeds 1482100–1482119, λ pinned); within-quintile tier permutation 10,000 draws + bootstrap 10,000 draws (seed 148240); covariate battery per the early-layer recipe at layer 20 (mean-to-spread ratio γ 0.642); cross-dictionary matching via chunked fp16 decoder GEMM (the 65,536² matrix never materialized)	matryoshka_tier/tier_tests.json; plan v21 §4

Evaluation: The per-context DV is the normalized squared error nerr(x) = ‖v̂(x) − v(x)‖² / ‖v(x) − μ_eval‖² (μ_eval = the eval set's mean answer state) — the exact per-context decomposition of the parent's pooled R² (1 − Σe2/Σdenom reproduces the whole-map R² identically); raw squared error and cos(v̂, v) are reported alongside. On the fresh 20k holdout the ridge whole-map R² is 0.7243 (mean cosine 0.9566). Two validity gates ran before any read. Reconciliation gate (PASS, no dropped arms): the full-train refits reproduce the committed aggregates on the pinned test — ridge 0.7541708 (|Δ| = 5.6e-16 against a 0.002 tolerance), mlp_w8192 |Δ| = 1.9e-9 and mlp_w32768 |Δ| = 1.3e-7 (0.01 tolerance), residual-skip 1.3e-9, KRR 6.6e-15; 13 fit units were seeded from the GCP partial attempt att-20260718-055220 and reconciled. SAE-fitness gate (PASS): on 247,819 pilot tokens of this run's own activations, fraction of variance explained 0.8097 at k=64 (the suite's published 0.8057) with L0 61.7, and 0.8453 at k=128; adjacent-layer probes localize the hook (layer 15 reads 0.533, layer 23 reads −9.32 — same-layer maximal, so no layer-indexing misalignment); the 8-row text→activation identity gate passed (min cos 0.9999). Judge instrument (labels are independent variables, not DVs): 19,967 of 20,000 holdout rows labeled; test-retest Cohen's κ on 200 re-judged items — language 0.991, topic 0.864, refusal-adjacency 0.901, answer-is-refusal 0.922, format 0.824 — all above the 0.6 demotion threshold. Primary criterion: Δ = mean nerr(non-English) − mean nerr(English) on the holdout-excluded ridge refit, judged by a two-sided 95% bootstrap CI against zero; a CI wholly below zero is the falsification outcome (an opposite-tail rejection is a legitimate directional finding, and the DV shows no floor/ceiling saturation — holdout nerr spans 0.019 to 2.48). Instrument-level control reads: the prefix-arm null maps read R² 0.0004 (feature space) and 0.0002 (dense) — on this single-turn corpus the prefix null and the train-mean baseline coincide; feature-space arm levels (context-features → answer-features ridge) are pooled R² 0.690 mean pooling / 0.359 max / 0.540 fraction-active with split-half rank stabilities 0.914 / 0.994 / 0.927; dense-input ridge reads 0.722 and dense-input MLP 0.739; the off-distribution encode-the-prediction read (1,448 features finite) correlates ρ = +0.17 with the direct feature map and stays secondary.

K-resample validity gates (all PASS, kresample/gates.json): the streamed parent-draw identity gate — squared error recomputed from the streamed v against the stored per-context error — reads median relative deviation 7.9e-05 (threshold 1e-3), Spearman 0.9999997 (threshold 0.999), and join-identity max relative deviation 4.2e-16; the subsample-coherence gate reproduces the full-holdout raw contrast (−0.0122, CI95 −0.0252 to +0.0010, against −0.0175); a draw-exchangeability diagnostic ranks the parent draw's error 2.989 of 5 on average (3.0 expected under exchangeability — the parent draw is distributionally unremarkable). Estimator-noise diagnostics: 0.7% of English and 0% of non-English contexts have a negative floor-adjusted error; floors are uncorrelated with answer length (Spearman −0.06 English, +0.00 non-English); 19 of 2,000 contexts have a zero floor (all four fresh draws identical). All 2,000 subsample rows joined; zero rows dropped.

Per-direction shrinkage-control gates (all PASS, perdirection-shrinkage-control/gates.json): the four split shas match the committed pins; the regenerated eigenvalues, the ridge per-direction R² at the parent λ, and the re-read MLP per-direction R² reproduce the committed values to median deviations near 1e-13 (thresholds 1e-3; rank agreement 1.0 against a 0.999 threshold). fp16-vs-fp64 scoring at the parent λ differs by at most 5.3e-6 mean absolute per-direction R² per band — four orders of magnitude below the band-4 gap, so the read is not precision-limited. Sensitivity and selection diagnostics: restricting selection to the 16-point core grid gives excess 0.1021; all 256 directions selected a λ off the parent value (127 heavier, 129 lighter; only 5.5% matched the holdout-oracle λ) with almost no holdout effect — 163 of 256 tuned directions score marginally below the committed arm, the self-protecting direction of selection noise.

Early-layer-arm gates (all PASS, early_layer/early_pilot.json): the public-dictionary availability/fitness gate reads fraction of variance explained 0.9204 at k=64 (pass threshold 0.825) with L0 67.6 on 32,040 pilot tokens of this run's own layer-3 activations, and 0.933 at k=128; layers 1 and 5 fail (0.7145 and −1.33), so layer 3 carries the arm. Hook-alignment probes are same-layer-maximal at both k; prefix-end constancy reads min cos 1.000; the 8-row capture identity gate reads min cos 0.99993 on early-layer states (bar 0.999) and 0.99988 flattened (bar 0.995). The layer-3 prefix-arm null reads pooled R² 0.0075 with retrieval at chance — the single-turn degenerate-prefix deviation carried again. Mapping baselines per fitted map: the layer-3 shared-feature identity+bias baseline (7,724 shared feature ids) reads pooled R² 0.080 with cosine retrieval acc@1 0.184 against the fitted map's 0.150 (chance 0.0002); the matched layer-19 identity+bias reads −0.046 and acc@1 0.022 against 0.183 fitted; the dense arms' identity baseline is inapplicable (3,584-dimensional input vs feature-space output).

Matryoshka-tier-round gates (matryoshka_tier/m_pilot.json; chat fitness verdict PASS, pile recorded OK — warn band, action proceed): the round-trip fitness gate reads fraction of variance explained 0.7971 (L0 103.5) for the chat-trained dictionary and 0.7087 (L0 107.9) for the pile-trained one on 32,040 pilot tokens of this run's own layer-20 activations — both above the 0.35 structural floor, no arm dropped; no published reconstruction figure exists for either dictionary, so the thresholds sit at the structural-breakage band, and our per-feature firing frequency correlates 0.76 with the repo's training sparsity table (an encode-convention fingerprint). Hook-alignment probes are layer-20-maximal (0.797, against 0.53 at layer 19 and 0.71 at layer 21); prefix-end constancy reads min cos 1.000; the 8-row capture identity gate against the parent's stored states reads min cos 0.99988 (flattened 0.99993). Both mapping arms ran per dictionary: the context arm reads pooled R² 0.646 (chat) / 0.649 (pile) with retrieval acc@1 0.551 / 0.435 (chance 0.00017); the prefix-arm null reads 0.013 / 0.016 with retrieval at chance — the single-turn degenerate-prefix deviation carried again; the dense-input companions read 0.663 / 0.675. The aligned-feature identity-plus-bias baseline (6,744 / 7,408 shared feature ids) reads pooled R² 0.084 / 0.085 with acc@1 0.12 / 0.13 — the fitted maps dominate both reads; the dense arms' identity baseline is inapplicable (3,584-dimensional input against feature-space output). Split-half rank stability reads 0.81 (chat) / 0.84 (pile).

Data extraction: Per-context error arrays join judge labels through the sha-verified scratch row map (19,967 of 20,000 joined). Bootstrap contrasts run as one masked matrix multiply over 10,000 draws. The script-intrusion scan matches CJK and Cyrillic codepoints in the judge-visible completion text (prompt capped at 1,500 chars, completion at 1,000 — 59.5% of completions at cap, so length strata reduce to three usable bins). The per-direction read projects the same holdout predictions as the per-context read onto the top-256 eigenvectors of the train-split covariance of v(x) (eigenvalues span 172 → 0.38, roughly 460×) and scores per-direction held-out R² for each fitter. Per-feature R² is feature-standardized over the 20k holdout, with split-half rank stability over two disjoint 10k halves. The k-resample floor for a context is the ddof-1 trace of the covariance of v over its four fresh draws divided by the parent's stored denominator; the adjusted contrast subtracts the arm-mean floor difference (subsample) from the raw full-holdout contrast, and the two bootstrap terms resample independently within arms (the ~10% subsample-within-holdout overlap makes them weakly dependent — the subsample-internal estimate is the stated sensitivity check). The shrinkage-control round scores all 38 λ values for all 256 directions off one eigendecomposition as per-λ blocked GEMMs; the tuned arm's holdout projections are gathered from retained per-λ fp16 projection tables, and the 10,000 bootstrap draws run as three GEMMs over per-context per-direction residual matrices. The early-layer arm pools per-token layer-3 SAE codes over the answer span under the reference inlier mask (BOS offset 8, 10× norm-outlier exclusion); the sink-robustness twin adds special-token positions to the mask; per-feature R², the shuffle null, and the covariate battery are computed from the same 6,000 score rows at both depths — the depth contrast holds rows, estimator, restriction recipe, and poolings fixed and varies only the layer. The matryoshka round encodes each token through both dictionaries in one capture pass (fp32, batched over tokens; per-token codes never materialized beyond the batch in flight), pools under the reference inlier mask plus the sink-robustness mask, and consolidates pooled rows into ~64 npz shards per family, uploaded before any fit. Cross-dictionary matching unit-normalizes decoder rows and reduces a chunked fp16 GEMM to each feature's best-match cosine in both directions; the tier permutation shuffles tier labels within activity quintiles and re-pools the Spearman statistic per draw.

Sample training/evaluation data + completions: Every answer is the parent round's single stochastic sample per prompt (Qwen-2.5-7B-Instruct, generation engine seed 42, max 1,024 new tokens) over first-turn LMSYS/WildChat user prompts; v(x) is the mean layer-19 activation over those answer tokens. Below: a random sample (seed 42) of 5 of the 19,967 labeled holdout rows — excerpts sanitized for this real-user-text corpus (mechanically truncated to ≤12 words); full text lives at eval_results/issue_1482/judge_labels/dispatch_main/dispatch_cbf37e0c4a0a/items.json in the pinned judge-labels tree, keyed by row id.

row id	judge labels	nerr	prompt excerpt (truncated)	model answer excerpt (truncated)
739609	en / factual Q&A / list	0.442	"how much urine can the average diaper hold?"	"The capacity of an average diaper can vary depending on the type"
420186	en / small talk / prose	0.237	"yaho"	a Chinese-script clarification request — the model answered in Chinese (the intrusion class)
628323	en / factual Q&A / prose	0.393	"Emma Stones surfing"	"Emma Stone is an Academy Award-winning American actress known for her roles"
85134	en / factual Q&A / list	0.521	"You are the text completion model and you must complete the assistant"	"3. Adhesion: The presence of water allows wet granular materials to adhere"
414734	en / coding / code	0.287	"Given a list of keywords: (Football, Stocks, HDFCBank, Valorant, …), generate their main categories, broad categories"	"Certainly! Here's the list of keywords with their main categories, broad categories,"

Worst-predicted-feature exhibits (cherry-picked worst tail of the 16,384 answer-side features; digests only — firing contexts are referenced by row id in the parent raw-completions chunks): feature 2651 (R² −3.4) fires on keyword-list/enumeration requests; 12614 (−2.7) on song-rewrite and ASCII-art creative prompts; 9325 (−2.4) on templated "write an article about" chemical prompts from WildChat; 63861 (−1.5) on competitive-programming asks.

Results

The error is category-structured: translation, NSFW, and harmful requests are the worst-predicted context types

Plotted: mean normalized prediction error per judge-labeled category on the fresh holdout (n = 19,967 labeled contexts; ridge, holdout-excluded refit) — the 12 topic classes in blue, the two language arms in orange, bars sorted, 95% bootstrap CIs; lower = better predicted.

Figure. Translation, NSFW, and harmful requests are the worst-predicted categories; non-English sits below English. Mean normalized error by judge-labeled category, ridge map, n = 19,967 holdout contexts (one sampled answer each). Orange: language arms (English n = 13,106, non-English n = 6,861); blue: the 12 topic classes. Error bars: 95% bootstrap CIs.

All 15 exploratory category contrasts clear BH-FDR at q=0.05 (10,000-draw permutation p), and task type spreads more widely than language: translation 0.337, NSFW 0.325, harmful requests 0.314, roleplay 0.310 at the top; small talk 0.235 and creative writing 0.257 at the bottom.

Two ranks are normalization-sensitive: small talk has the largest answer spread (denominator 2,008) with above-average raw error, so its best rank is partly normalization-assisted; coding is the mirror image. Refusal-adjacent requests read 0.320 vs 0.282 elsewhere, and their penalty is largest in the longest answer stratum, which argues against a short-answer artifact.

Underlying data: the same per-context errors as jittered points per category row, group means marked.

Figure. The per-context error distributions behind the category bars. One point per labeled holdout context in each category row it belongs to (topics blue, language arms orange; log x); black diamonds mark the group means plotted as bars above. n = 19,967.

Non-English contexts are predicted better than English, falsifying the planned direction of the language contrast

Plotted: per-language mean normalized error against that language's holdout count (log x), one labeled point per judge-detected language with n ≥ 30, 95% bootstrap CI whiskers.

Figure. Most non-English languages cluster below English; Arabic and Polish are the clear exceptions. Each point is one language (17 of 51 detected have n ≥ 30); x = labeled holdout contexts in that language (log scale), y = mean normalized error ± 95% CI. English (n = 13,106) is the rightmost point.

The planned contrast Δ = mean nerr(non-English) − mean nerr(English) is −0.0175 (CI95 −0.0221 to −0.0129; n = 6,861 vs 13,106); the CI is entirely below zero, so the planned direction is falsified. Raw squared error agrees (−28.1, CI95 −35.0 to −20.9); arm denominators match to 0.08%. Only the uncentered cosine flips (0.9555 vs 0.9572).

The binary split hides most of the structure: per-language means run from German at 0.236 to Arabic at 0.420, ten times the binary gap. Chinese (0.249) and French (0.257) sit near the German end, Polish (0.376) and Farsi (0.336) near the Arabic end. Chinese alone carries a third of the non-English arm.

Train frequency shows no detectable gradient (Spearman(per-language n, error) = −0.08, p = 0.75; underpowered at 17 languages), though the sign cuts against the underrepresentation account: the rarer arm is predicted better.

The language advantage survives corpus transfer and intrusion exclusion

Plotted: mean normalized holdout error for four cells — mixed vs LMSYS-only training, scored per corpus half of the holdout (95% CIs).

Figure. The non-English advantage persists when the map is fit on LMSYS alone. Ridge map refit on mixed train vs LMSYS-only train, each evaluated per corpus half of the 20k holdout. Error bars: 95% bootstrap CIs.

Fitting only LMSYS costs nothing in-corpus (0.290 vs 0.293) but +0.061 on WildChat (0.337 vs 0.276).

Under the transfer fold Δ stays −0.0149 (CI95 −0.0196 to −0.0102), so the language read is not within-corpus structure. Script intrusion does not carry it: 226 of 13,106 English-arm answers show strict CJK intrusion (1.7%; Cyrillic adds 41); intruded rows are predicted worse (0.392 vs 0.290), yet excluding them leaves Δ = −0.0157. The 2,874 CJK-bearing non-English completions are in-script and were kept.

One alternative remained: with one sampled answer per context, English prompts eliciting higher-entropy answers could have produced part of the gap. The resampling probe in the next result rules that out — the sampling floor is higher in the non-English arm.

Underlying data: per-context errors behind the four cells, cell means marked.

Figure. Per-context error distributions behind the four transfer cells. One point per holdout context under each train-corpus and holdout-corpus pairing (mixed train blue, LMSYS-only orange; log y); black diamonds mark the cell means. n = 10,983 LMSYS and 9,017 WildChat contexts per refit.

The answer-sampling floor is higher for non-English contexts, so the floor-adjusted language advantage widens

Plotted: per-arm decomposition of expected per-context error into map share plus answer-sampling floor, and the raw, floor, and adjusted non-English-minus-English contrasts with 95% bootstrap CIs; 2,000-context stratified subsample of the holdout; the plotted floor is the fresh-4 estimator (draws 43–46; the parent draw 42 completes the five-draw shadow).

Figure. Non-English answers vary more under resampling, yet remain better predicted: floor adjustment widens the advantage. Left: mean floor-adjusted map error stacked under the mean answer-sampling floor, per language arm (1,000 contexts each, 5 answer draws). Right: the raw, floor, and adjusted contrasts, 95% bootstrap CIs.

Four fresh answers per context (seeds 43–46, the parent decoding recipe) plus the parent's own draw measure the within-context sampling variance of v(x). The floor is a third of mean error in the English arm and 40% in the non-English arm (0.094 vs 0.110; floor difference +0.017, CI95 +0.008 to +0.025) — the entropy account predicted the opposite ordering.

Subtracting it widens the contrast from −0.017 to −0.034 (CI95 −0.044 to −0.024); the subsample-internal estimate (−0.032) and raw squared-error units agree. Under the MLP fitter the raw gap is smaller (−0.005) but the adjusted gap stays negative (−0.022, CI95 −0.032 to −0.012).

Underlying data: per-context floors against the stored single-draw error.

Figure. Per-context floors span two orders of magnitude and rise with stored error in both arms. One point per subsample context: the four-fresh-draw floor estimate against the parent's stored normalized error (log-log; English blue, non-English orange). 1,981 of 2,000 contexts shown — 19 with identical draws (zero floor) cannot render on the log axis.

Nonlinearity's advantage is broad across contexts and stable across seeds and widths

Plotted: per-context normalized error under the ridge (x) against the width-8,192 MLP (y) on the 20k holdout, log-log; points below the diagonal are contexts the nonlinear map predicts better.

Figure. Most contexts sit on the ridge-worse side of the diagonal: the nonlinear map is better almost everywhere. Paired per-context errors, ridge vs MLP (width 8,192, seed 0), n = 20,000 holdout contexts; both fitters score the same sampled answer, so the pairing removes answer-entropy differences.

The MLP improves 90.7% of contexts (mean gap 0.0547 normalized-error units). Seed noise does not reproduce this pattern: the per-context ranking replicates across MLP seeds (Pearson r = 0.915; top-decile context-set overlap 80.4%, Jaccard 0.672) and across widths (the width-32,768 fitter is better on 88.4% of contexts, gap correlation 0.90). Per-category mean gap is nearly flat, 0.045 (small talk) to 0.073 (NSFW), so no single category drives it.

The per-context gap is less concentrated than pure MLP seed noise

Plotted: cumulative share of total |gap| against the fraction of holdout contexts sorted by |gap| (a Lorenz-style curve), with the MLP seed-to-seed difference as the noise reference and the uniform line.

Figure. The gap curve sits closer to uniform than the seed-noise reference. Cumulative |gap| share vs context fraction, n = 20,000. Curves: the ridge-vs-MLP gap, the MLP seed-to-seed difference (noise reference), and the perfectly-uniform line.

The top decile of |gap| carries 30.1% of the total, below the pure seed-noise reference (38.5%), while the total gap mass is 4.0 times the seed-noise total (1,237 vs 308). Nonlinearity buys a small improvement almost everywhere rather than rescuing a failing tail. Of the competing hypotheses, this supports spread-across-contexts over concentrated-in-tail-contexts.

In representation space the gap is structured: it grows about fourfold toward low-variance directions of the answer state

Plotted: per-direction held-out R² for ridge and MLP across the top-256 PCA directions of the answer state (left half of the figure), and their per-direction gap with band means (right half); directions ranked by train-split variance.

Figure. The ridge-vs-MLP gap widens steadily toward low-variance PCA directions. Same holdout predictions as the per-context read, projected onto the top-256 train-covariance PCA directions of v(x). Left: per-direction R², ridge vs MLP. Right: their gap, with means over the four rank bands.

Band means over ranks 1–16/17–64/65–128/129–256: ridge 0.865/0.707/0.562/0.416, MLP 0.900/0.786/0.677/0.554, gap 0.035/0.079/0.116/0.138. The gap rises monotonically across the four bands, 3.9× top to bottom (correlation with rank: 0.87 Pearson, 0.91 Spearman), though the per-direction series is noisy (122 of 255 adjacent steps decrease). The MLP wins on all 256 directions (min gap +0.017).

The dominant directions are close to linear; most of the nonlinear advantage sits in low-variance fine structure. Caveat: one MLP seed (seed 0) at the direction level (the seed-stability read is per-context). The shrinkage alternative — a shared ridge penalty over-shrinking low-variance directions — is adjudicated and closed by the next result.

Per-direction ridge tuning closes none of the low-variance gap: the concentration is genuine nonlinearity, not differential shrinkage

Plotted: per-direction held-out R² across the top-256 answer-PCA ranks — committed shared-regularizer ridge, this round's per-direction-tuned ridge, committed MLP — with band-mean gap bars inset; same holdout predictions, split, and basis as the previous result.

Figure. The per-direction-tuned ridge overlays the shared-regularizer ridge at every rank; the MLP stays above both. Per-direction held-out R² vs answer-PCA rank (n = 20,000 holdout contexts); inset: band-mean MLP−ridge gaps under both regularizer conventions. λ selected per direction on the parent's pinned 400-row val over a 38-value grid.

Per-direction tuning leaves the gap excess (band 129–256 minus band 1–16) at 0.1030 (CI95 0.1013 to 0.1046) against 0.1021 committed; the holdout-oracle λ — the ceiling of any grid-linear per-direction baseline — reads 0.1020, excluding selection noise. The outcome was analytically forced: against the smallest train-Gram eigenvalue (2,447) the parent λ of 0.001 implies shrinkage factors at least 0.99999959 — the committed ridge was already numerically an unshrunk least-squares fit.

Tuned band gaps 0.036/0.079/0.116/0.139 preserve the fourfold rise; the MLP stays better on all 256 directions. This closes the previous result's shrinkage caveat.

Underlying data: the 256 per-direction gaps under both regularizer conventions.

Figure. All 256 directions hug the identity line: per-direction tuning barely moves any gap. MLP−ridge gap under the per-direction regularizer against the shared-regularizer gap, one point per direction, colored by rank band; dashed identity. Low-variance directions sit slightly above the line — tuning widened their gaps marginally.

Which answer-side SAE features are predictable is an answer-side property, and rare features are worst-predicted

Plotted: per-feature held-out R² (clipped at −1) against feature activity (fraction of the 120k fit contexts where the feature is active, log x) — one point per answer-side feature (16,384), decile medians overlaid, the 30 worst-predicted features marked.

Figure. Rare answer-side features are worst-predicted; the decile median rises with activity. Context-features → answer-features ridge map (mean pooling), 20k holdout. One point per answer-side SAE feature; line: median R² per activity decile; marked points: the 30 worst-predicted features.

Median per-feature R² rises near-monotonically with activity, from 0.091 (lowest decile) to 0.266 (top; one dip, 0.1058→0.1050), and the share below zero falls from 16.5% to 1.6%. The overall median is only ~0.13, so the pooled R² of 0.690 rides on high-variance features.

Which features are predictable barely depends on the input representation (Spearman 0.93 between the context-features and dense-input ridge rankings): failures are answer-side properties. Worst-tail exhibits cluster on task formats, and 290 above-median-activity features still read below zero — the label-shuffle null in the next result adjudicates this tail.

The SAE→SAE MLP fits diverged (pooled R² −1e10 to −6e13), so the feature-space linear-vs-nonlinear read rests on the dense-input arm, where it is small (+0.017).

A label-shuffle null clears nearly all answer-side features above the estimation-noise floor: negative per-feature R² is mostly mechanical

Plotted: the same per-feature held-out R² against feature activity as the previous result (one point per answer-side feature, log x, clipped at −1), overlaid with the K = 20 label-shuffle null — the per-activity-decile pooled 2.5–97.5% null band, the null median, and the observed decile median.

Figure. The shuffle-null band sits wholly below zero; the observed decile median stays above it at every activity level. Same read as the previous figure, with K = 20 label-shuffle draws (answer rows permuted within the 120k fit set, λ pinned at 10,000, scored on the untouched 20k holdout) pooled per activity decile. n = 16,384 features.

The null is itself negative everywhere (median ≈ −0.07, decile q2.5 down to −0.139), so a negative observed R² is not by itself anomalous. Against each feature's own K = 20 band, 98.2% of the 16,384 features sit above chance and 128 (0.8%) fall below — a read whose leave-one-out false-positive rate is 6.3%, so the coarse per-feature bands show no detectable excess of worse-than-chance features overall.

Of the 1,327 observed-negative features, 1,031 still beat their band; of the 290 above-median-activity negatives, 36 (12%) fall below theirs, about twice the calibration rate — a small genuinely anomalous residue. The identity draw reproduced the parent's per-feature R² (median |Δ| = 1.5e-5); the decile-pooled bands are the better-calibrated aggregate read.

Within-answer consistency, not rarity, is the dominant per-feature predictability correlate

Plotted: left — the same per-feature held-out R² (clipped at −1) against within-answer consistency (mean fraction of the answer's tokens where the feature is active, conditional on being answer-active; 120k fit rows), log x, one point per answer-side feature (16,384), decile medians overlaid; right — median per-feature R² per consistency tercile within each activity decile.

Figure. Consistently-firing features are well predicted, and the effect holds inside every activity decile. Context-features → answer-features ridge map (mean pooling), 20k holdout; consistency accumulated over the 120k fit rows (recomputed activity matched the committed covariate exactly).

Spearman(consistency, R²) = 0.60 — twice the activity correlate (0.29) — and it is not an activity proxy: ρ(consistency, activity) = 0.18 and the rank-partial correlation given activity stays 0.58; an answers-≥8-tokens robustness cut reads 0.60. The high-consistency tercile's median R² exceeds the low tercile in all 10 activity deciles. Median consistency is 0.007, so most features are phasic — active on a few answer tokens — and a phasic feature's mean-pooled target is dominated by exactly where it fires, which the context state evidently does not determine.

Underlying data: per-feature arrays in eval_results/issue_1482/feature_correlates/consistency_perfeature.npz.

Judged feature abstraction level does not predict per-feature R²

Plotted: left — per-feature held-out R² by judged level (low = surface/token/format property; high = semantic/abstract property; unclear), one point per sampled feature (n = 300, stratified activity decile × within-decile R² tercile), black bars = medians; right — median R² for low vs high level within each activity decile.

Figure. High- and low-level features are equally predictable; the within-decile lines interleave. Labels: project judge over each feature's top-8 firing answers plus the Neuronpedia trainer_1 auto-interp description as auxiliary evidence (test-retest κ = 0.619, 0 drops).

High-level features read median R² 0.156 (CI95 0.126–0.196) vs low-level 0.136 (0.109–0.174), statistically indistinguishable (p = 0.57); Spearman(level, R²) = 0.04 (0.05 given activity). The instrument is only moderately reliable (κ = 0.619) and the evidence is answer-level snippets plus a pile-corpus auto-interp, so a small effect could hide under label noise — but the point estimate is near zero while the same 300 sampled features carry the consistency signal (ρ(level, consistency) = 0.16).

Unclear-labeled features (n = 45) trend worse (median 0.098). The extremes follow-up refines this: at the R² tails, high-level features concentrate among the best-predicted (52.1% vs 24.8%, p < 1e-5).

Underlying data: per-feature labels + stats in eval_results/issue_1482/feature_correlates/abstraction.json.

Better-predicted answer-side SAE features are more r_B-aligned; the judged persona contrast is carried by language and register, not identity

Plotted: left — fraction judged persona-related (who is speaking / persistent manner of speaking, vs topic or format), for global R² tails (150+150) and activity-decile-matched tails (15+15 per decile); right — all 16,384 features: max-over-traits |cos(SAE decoder, r_B)| (mean-difference trait directions, layer 19; provenance in footer) vs per-feature R², decile medians.

Figure. Persona-aligned features are better predicted; two independent methods agree. Judged persona rate 20.3% best vs 3.0% worst (OR 8.33, decile-stratified permutation p < 1e-4; Set A 20.8% vs 5.1%). Mechanical alignment vs R²: Spearman +0.203 (+0.243 given activity). 358/358 labeled, 0 drops.

The judge-free read stands alone: decoder–r_B cosine tracks per-feature R² (Spearman +0.203, +0.243 given activity; the 50 most-aligned features sit at the 92.8th R² percentile vs median 0.146); median max|cos| 0.035 is directional, not a subspace.

The judged persona contrast (20.3% vs 3.0%, OR 8.33) does not survive decomposition: the 40 persona-labelled features split 20 language / 11 register-style / 9 identity-disposition, and on the identity subset it vanishes (2/150 best vs 4/150 worst). The rubric counted language and register as persona, and answer language is near-deterministic from the query: a manner-of-speaking effect, not speaker identity. The judged field is per-feature unreliable (κ = 0.14).

Dashboard + data: tasks/.../artifacts/feature_extremes_dashboard.html, eval_results/issue_1482/feature_extremes/ @ fcc9965459.

Token-level layer-3 features are nearly as predictable as layer-19 features: the depth profile shifts down slightly and does not invert

Plotted: ECDFs of per-feature held-out R² (clipped at −1) from the matched-row refits (24,000 fit / 6,000 score contexts, 16,384 features per depth, same ridge recipe): layer 3 vs layer 19; layer-3 features by dense-latent flag; layer-3 features by direct-logit-footprint tercile.

Figure. Layer 3 tracks layer 19 with a modest downward shift; dense latents are better predicted; footprint terciles barely separate. Per-feature held-out R² ECDFs on matched rows (24,000 fit / 6,000 score; 16,384 features per depth). Middle: sparse (n = 14,977) vs dense (n = 1,407, activity above 0.5) layer-3 latents.

Median per-feature R² is 0.095 at layer 3 vs 0.129 at layer 19 (88.2% vs 93.2% positive), 92.3% of layer-3 features clear the shuffle-null 97.5th percentile, and pooled R² orders the same way (0.474 vs 0.639 context-features; 0.473 vs 0.700 dense-input). Extending the depth range to the token stratum does not invert or collapse the profile.

Dense latents are better predicted at both depths (layer-3 median 0.245 vs 0.095 overall). The layer-3 dense-input MLP scores below the ridge (0.326 vs 0.473 pooled) at this 24,000-row fit size, so the nonlinear advantage cannot be assessed here; judged abstraction-level tests are deferred (labels frozen).

Underlying data: per-feature points behind the layer-3 curve.

Figure. The layer-3 activity gradient mirrors layer 19's: rarer features are worse predicted. One point per layer-3 answer-side feature (16,384); line: median R² per activity decile; dashed: the K = 20 shuffle-null 97.5th percentile.

Within-answer consistency dominates layer-3 predictability too, and even more strongly than at layer 19

Plotted: per-feature held-out R² (clipped at −1) against within-answer consistency (mean fraction of the answer's tokens where the feature is active, conditional on being answer-active; 24,000 fit rows), one point per layer-3 answer-side feature (16,384).

Figure. Consistently-firing layer-3 features are well predicted; near-zero-consistency features carry the negative tail. Layer-3 context-features → answer-features ridge map, 6,000 score rows; consistency accumulated over the 24,000 fit rows.

Spearman(consistency, R²) = 0.68 at layer 3 — above the layer-19 read on the same rows (0.56) — with the rank-partial given activity at 0.62; activity itself reads 0.39. Median layer-3 consistency is 0.007, so the token stratum is at least as phasic as layer 19, and the same mechanism carries: a feature whose within-answer firing pattern is stable is predictable from the context state.

The direct-logit footprint correlate is weak at layer 3 (0.14; partial 0.09 given consistency) and absent at layer 19 (−0.05); the co-activation covariate saturated at 1.0 for over 99.9% of features (uninformative — see the scope caveat). Underlying data: per-feature arrays in eval_results/issue_1482/early_layer/covariates_l3.npz and covariates_l19.npz, and the footprint per-feature scatter below.

Figure. Footprint concentration barely orders layer-3 predictability. One point per layer-3 answer-side feature (16,384); x = the fraction of the feature's direct-logit mass on its top-20 vocabulary tokens.

Persona-direction decoder alignment tracks predictability at layer 19 but not at layer 3

Plotted: per-feature held-out R² (clipped at −1) against max-over-traits absolute cosine between the feature's decoder and the persona-trait mean-difference directions (population-centered), one point per layer-3 answer-side feature (16,384).

Figure. No alignment gradient at the token stratum. One point per layer-3 answer-side feature; x = max-over-traits |cos(decoder, persona direction)| after population centering, traits evil / sycophancy / hallucination at the matched layer.

On the matched rows the layer-19 read reproduces the committed finding — decoder–persona-direction alignment tracks per-feature R² at ρ = 0.18 (0.19 raw) — while layer 3 reads ρ = 0.06 (0.05 raw). The persona-alignment correlate is therefore depth-specific, consistent with trait directions living at mid-depth rather than in token-identity space; the earlier extremes result's persona-alignment gradient is a property of the post-detokenization stratum, not of SAE features generally.

Both depths use the same trait bank at their own layer, the same restriction, and the same score rows, so the contrast is not a recipe artifact.

Sink masking is a no-op at layer 3 and a second dictionary replicates the profile

Plotted: per-feature held-out R² under the sink-robustness pooling mask (reference mask plus special-token positions) against the default reference mask, one point per layer-3 feature (16,384), identity line dashed.

Figure. The two pooling masks agree essentially exactly. Paired per-feature R² under the two answer-pooling masks, layer-3 context-features map, 6,000 score rows; dashed identity line.

The paired masks agree to Spearman 0.9999993 with median absolute difference 3.2e-6 — attention-sink positions carry none of the layer-3 read (the reference mask's BOS-offset + norm-outlier exclusion already covers them). The independent k=128 dictionary — a different 16,384-feature population from a second trainer — reproduces the profile: median R² 0.119, 91.2% positive, pooled 0.516, slightly above k=64 in line with its higher reconstruction quality (0.933 vs 0.920). The layer-3 findings are not artifacts of one dictionary or of sink handling.

Underlying data: the second-dictionary distribution against the primary.

Figure. The second dictionary's ECDF overlays the first's. Per-feature held-out R² (clipped at −1), k=64 vs k=128 layer-3 dictionaries, 16,384 features each, same fit and score rows.

Training-imposed granularity predicts predictability: coarse matryoshka features are predicted better, beyond their firing frequency

Plotted: per-feature held-out R² (median with interquartile bars) of the layer-20 context-features map, by matryoshka tier within activity quintile — chat-trained dictionary in the left half, pile-trained in the right; 16,384-feature panels; the label-shuffle null's 97.5th percentile dashed.

Figure. General-tier features are predicted better than specific-tier ones in the upper three activity quintiles; the ordering reverses in the two lowest, where the general-tier median is lowest. Median ± IQR per-feature R², 6,000 held-out rows; activity quintiles are the plan's fallback from deciles (sparsest decile-tier cell held 17 features).

The matryoshka objective trains feature ids 0–2,047 to reconstruct alone, so tier is a training-imposed, judge-free granularity label (zero judge calls this round). Raw tier medians fall 0.435 → 0.174 → 0.043, and the pooled Spearman between tier and R², −0.395, lies below the −0.250 to −0.228 band from 10,000 tier-label permutations within activity quintiles — the coarse-better gradient exceeds what the tier–activity confound alone produces (partial Spearman −0.194 given log activity). The context arm carries the signal (pooled R² 0.646) against a prefix-arm null of 0.013.

The per-feature scatter behind the tier profile: activity and tier jointly organize layer-20 predictability

Plotted: one point per chat-dictionary panel feature (16,384) — held-out R² (clipped at −1) against answer-side activity (log scale), colored by tier; label-shuffle null 97.5th percentile dashed.

Figure. At matched activity, general-tier points sit above mid- and specific-tier points across the upper activity range. 16,384 features, 6,000 held-out rows; R² clipped at −1 for display.

Predictability rises with activity inside every tier, and the tier separation is visible at matched activity across the upper quintiles. In the two lowest-activity quintiles the ordering reverses — the general-tier median is lowest (−0.11, then 0.00) — and the gradient strengthens with activity above them (within-quintile Spearman +0.07 in the lowest quintile to −0.53 in the highest). 79.6% of chat-dictionary panel features clear the shuffle-null 97.5th percentile (83.8% for the pile panel), and split-half rank stability reads 0.81 (0.84 pile) — the per-feature ranking the tier read pools is mostly signal, not estimation noise.

The pile-trained twin reconstructs chat worse, yet its features are slightly more predictable

Plotted: median per-feature held-out R² by tier for both dictionaries (left half), and each dictionary's round-trip reconstruction quality (fraction of variance explained) on this corpus's layer-20 tokens (right half).

Figure. A 0.09 reconstruction gap does not become a predictability deficit. Medians over 16,384-feature panels (left half); round-trip reconstruction on 32,040 pilot tokens (right half).

Both dictionaries cleared the 0.35 fitness floor (0.797 chat, 0.709 pile); the 0.088 gap is itself the corpus-dependence finding. The predictability half reversed its planned direction: pile features read slightly higher median R² (0.133 vs 0.116; chat-minus-pile CI95 −0.023 to −0.010), driven by the mid and specific tiers; only tier 0 favors chat (CI95 +0.026 to +0.063).

The comparison is unpaired — each panel keeps its own activity-floor-clearing features (pile 10,958 at tier 2 vs 12,125 chat) — so selection on chat-transferability can contribute to the reversal. Retrieval localizes the pile deficit input-side: context-arm acc@1 0.435 vs 0.551 chat at similar pooled R², reversed on dense-input arms (0.526 vs 0.489). Both dictionaries show the same coarse-better tier gradient.

Underlying data: per-feature R² distributions behind the medians.

Figure. The pile panel's higher R² holds across most of the distribution, reversing only within tier 0. Cumulative distribution of per-feature held-out R² (clipped at −1 for display), pooled (left half) and by tier (right half); solid = chat-trained, dashed = pile-trained; 16,384 features per panel, 6,000 held-out rows; unpaired — each panel keeps its own floor-clearing features.

Coarse matryoshka features are shared across training corpora; specific ones are corpus-bound

Plotted: the cumulative distribution of each feature's best cross-dictionary decoder-cosine match, per tier, over all 65,536 features — chat→pile matches in the left half, pile→chat in the right.

Figure. Tier-0 curves sit far right of tier-2 curves in both match directions. Unit-normalized decoder rows; one best-match cosine per feature.

Tier-0 features find a counterpart in the other dictionary at median cosine 0.57 (33% match at 0.7 or above); tier-2 features read median 0.08 (3.3% at 0.7 or above). The general subspace is largely corpus-independent while specific features are corpus-bound — consistent with the tier gradient: what the context→answer map predicts well is also what survives swapping the dictionary's training corpus. The mechanical persona-presence read is a bounded null rather than evidence of absence: essentially no chat-dictionary feature is meaningfully persona-aligned (the top-decile alignment cutoff is absolute cosine 0.05), and that weakly-aligned decile matches pile at median cosine 0.390 against a population median of 0.425 (judged confirmation stays frozen).

Within-answer consistency again dominates layer-20 predictability, and the tier read is unchanged under sink masking and scoring corpus

Plotted: per-feature held-out R² against within-answer consistency (left half) and against the population-centered maximum decoder–persona-direction cosine (right half), chat dictionary, one point per panel feature, colored by tier.

Figure. Consistency organizes the predictable tail; persona-direction alignment barely does. 16,384 features, 6,000 held-out rows.

Consistency correlates ρ = 0.60 with per-feature R² (0.59 pile; partial 0.52 given activity) — the third depth-and-dictionary regime where this covariate dominates. Persona-direction alignment stays weak (ρ 0.13 chat / 0.10 pile), sitting between the earlier depth reads (qualitative comparison — different dictionaries). Dense-flag features (about 11% of each panel) concentrate in tier 0.

Two nuisance checks are clean: the sink-robustness pooling twin is a no-op (paired Spearman 0.99998, median absolute difference 6e-6), and the tier profile is preserved when scoring on LMSYS-only against WildChat-only halves of the held-out rows, with WildChat medians slightly higher at every tier.

---

Repro: Compute: ~30–34 realized GPU-h (plan headline 12; three GCP a2-ultragpu-4g attempts — partials retained at HF issue1482_partial/att-20260718-{015406,055220,111442} @ cbc55efdd7 as forensic/seed records — then a RunPod 4×H100 failover pod completed the pod phases; judge + analysis ran off-pod); k-resample round ~3.1 realized GPU-h of 4 approved (a GCP A100 crash, a RunPod 4×H100 run under a mismatched recapture convention, then the GCE A100-80 re-run under the parent convention); per-direction shrinkage-control round ~1.3 realized GPU-h of 3 approved (one GCE A100-80). Code: branch issue-1482 at 43818beb136bac133e33414a6520760b4206afc6 — driver scripts/issue1482_error_analysis.py, SAE loader scripts/issue1482_sae.py, analysis scripts/issue1482_analysis.py; follow-up shuffle-null scripts/issue1482_shuffle_null.py (main @ 43bbbbb0ed); k-resample driver scripts/issue1482_kresample.py (branch run at 7f8b47c81d, results committed at 01316e66cd); k-resample body figures scripts/issue1482_kresample_figs.py (1f0bc2a174); per-direction shrinkage-control phase in scripts/issue1482_error_analysis.py at f9085be7bf (results + figures committed at 27a49e79a1). Eval artifacts (git, pinned): eval_results/issue_1482 tree — percontext/ (13 JSON summaries; the 13 per-context npz arrays are on HF), reconciliation.json, sae_fitness.json, sae_perfeature/, judge_labels/, taxonomy.json, h1_contrast.json, gap_decomposition.json, perdirection_pca.json, split_1482.json; follow-up round sae_perfeature/shuffle_null.json + sae_perfeature/shuffle_null_perfeature.npz @ 43bbbbb0ed; k-resample round eval_results/issue_1482/kresample tree — gates.json, floor_summary.json, adjusted_contrast.json, strata.json, percontext_floor.npz; per-direction shrinkage-control round eval_results/issue_1482/perdirection-shrinkage-control tree — gates.json, pdshrink_summary.json (the per-direction λ and R² tables lam_tables.npz plus the tuned holdout projections are on HF). HF data repo (listed live at write time): issue1482_error_analysis/analysis_tensors — sae_pooled/ (1,920 npz), percontext/ (26 files), scratch_meta/; k-resample round issue1482_kresample — raw_completions/ (rollout text, all four fresh seeds), analysis_tensors/{V.npz,capture_meta.json,percontext_floor.npz,v42_stream.npz,v42_stream_meta.json}, inputs/; per-direction shrinkage-control round issue1482_error_analysis/analysis_tensors/pdshrink — lam_tables.npz, tuned_holdout_projected.npz, summary + gates + figure mirrors (listed live at write time). Figures on main, pinned at f1947c38acbe and bdabd70d30e7 (PNG + PDF + data sidecars); the two per-context companion figures and the regenerated shuffle-null figure at 009a1d13f5 (PNG + PDF + data sidecars); the two k-resample body figures on branch issue-1482 at 1f0bc2a174 (PNG + PDF + data sidecars), with the driver's diagnostic renders (per-language floors, identity-gate and rank views, unnormalized twin, MLP comparison) at 01316e66cd; the three per-direction shrinkage-control figures (hero, per-direction gap scatter, and the selection, grid, and shrinkage-factor diagnostics) on branch issue-1482 at 27a49e79a1 (PNG + PDF + data sidecars). Condition-name → artifact-filename legend: full-train reconciliation refit refit_full; holdout-excluded refit refit_holdout (second MLP seed refit_holdout_seed43); LMSYS-only transfer refit refit_lmsys_transfer; context-features map sae_ctx; dense-input feature map sae_dense_in; prefix-arm null sae_prefix_null; encode-the-prediction read sae_encode_pred; SAE fitness pilot sae_fitness; split-half stability read sae_perfeature/summary.json; per-feature shuffle null sae_perfeature/shuffle_null.json; k-resample floor arms (kres_en / kres_nonen) kresample/floor_summary.json; adjusted contrast plus the MLP and raw-units re-reads (kres_mlp, kres_raw_units) kresample/adjusted_contrast.json; per-direction shrinkage-control arms (val-tuned and holdout-oracle ridge) perdirection-shrinkage-control/pdshrink_summary.json. Early-layer-arm round (2026-07-29): ~4.6 realized GPU-h of 4 approved on one GCP flex-start A100-80 (final leg att-20260729-142155, pod wall 4.58 h — pilot 53 s, capture 96 min, store upload 72 min, fits 106 min per early_layer/phase_times.json; a same-day first leg was superseded by an instance restart and the final leg re-ran capture through upload end to end); driver scripts/issue1482_early_layer.py, launcher scripts/issue1482_early_layer_launch.sh, layer-generalized loader scripts/issue1482_sae.py, all on branch issue-1482-earlylayer at 5fd1ee6b8dcb0e5d41f016cefb6be85b5683e6ba; eval artifacts (git, pinned): eval_results/issue_1482/early_layer tree — early_pilot.json, early_summary.json, split_early.json, mech_tests.json, phase_times.json, 18 per-feature npz (both depths, three poolings), covariates_l3.npz, covariates_l19.npz, shuffle_null_l3.npz, and evidence/evidence.json (the 377-feature labelling-resume packet); HF data repo (listed live at write time): issue1482_error_analysis/analysis_tensors/early_layer — store/ (1,920 pooled_l3 shards + 1,920 dense_l3 shards + split_indices_early.npz + regime.json), eval/ (25 files), e5_work/ (2 files); the seven early-layer figures on branch issue-1482-earlylayer at 5fd1ee6b8d (PNG + PDF + data sidecars). Early-layer condition legend: layer-3 context-features map perfeature_l3_default (poolings _max / _frac); sink-robustness twin perfeature_l3_sinkmask; second dictionary perfeature_l3_k128; layer-3 dense-input map perfeature_l3_dense_in (MLP twin _mlp); layer-3 prefix null perfeature_l3_prefix_null; matched-row layer-19 controls perfeature_l19_matched_ctx / perfeature_l19_matched_dense. Matryoshka-tier round (2026-07-30): ~5.6 realized GPU-h of 6 approved on one GCP flex-start A100-80 (eps-issue-1482, pod phases 02:29–08:07Z — pilot 43 s, capture 79 min, store upload 124 min, fits 117 min per matryoshka_tier/phase_times.json; the evidence + analysis phases ran off-pod on the VM); driver scripts/issue1482_matryoshka_tier.py, launcher scripts/issue1482_matryoshka_launch.sh, and the SAELensJumpReLU loader in scripts/issue1482_sae.py on branch issue-1482-matryoshka at cc1109ba5826d8dfea2a330c44ee04e099ba57e8 (code), eval artifacts committed at f4f3553df0e05902a3347f4508635dd29deaebf3, and the analysis harvest (tier tests, evidence, figures) at a65efab7dfaec7fbc1991c60d38d5883c53b5514; eval artifacts (git, pinned): eval_results/issue_1482/matryoshka_tier tree — m_summary.json, m_pilot.json, m_split.json, m_panel.json, tier_tests.json, phase_times.json, evidence/evidence.json (540 label-free evidence packets preserving the judged resume spec), and the per-feature / covariate / population / shuffle-null / cross-matching npz arrays; HF data repo (listed live at write time, revision 8b82eb975326774003512db1e7e542491edcd056): issue1482_error_analysis/analysis_tensors/matryoshka_tier — store/ (194 files: pooled layer-20 shards for both dictionaries + split indices + regime) and eval/ (the 29-file git mirror); upload-verification v9 PASS. The nine matryoshka figures on branch issue-1482-matryoshka at a65efab7df (PNG + PDF + data sidecars); the pile-vs-chat per-feature R² ECDF companion at adb1d02191 (PNG + PDF + data sidecar; rendered from the committed perfeature_m_lmsys_default.npz + perfeature_m_pile_default.npz). Matryoshka condition legend: chat-dictionary map perfeature_m_lmsys_default (poolings _max / _frac; sink twin _sinkmask*); pile-dictionary map perfeature_m_pile_default; dense companions perfeature_m_lmsys_dense_in* / perfeature_m_pile_dense_in*; prefix nulls perfeature_m_lmsys_prefix_null / perfeature_m_pile_prefix_null; statistical tests tier_tests.json.

• Reused captures + rollout text + manifest from #779: HF issue779_monitoring/fitter-fair-comparison-n1m/{final_token_capture (1,920 chunks), raw_completions (1,936 chunks), sampling_manifest} @ cbc55efdd7 — fit: same map, same layer, pinned split sha-asserted in-driver; refits reconcile to the committed aggregates within |Δ| ≤ 1.3e-7.
• Reused fitters from #779: scripts/issue779_ffc_n1m_fits.py executed unchanged from the merged issue-779-n1m branch tip d7c1c55fbe (fit commit bd9f6865de16) — fit: seed 0 and the explicit 16,384 KRR centers match the parent record; the reconciliation gate adjudicated numeric equivalence.
• Reused SAE andyrdt/saes-qwen2.5-7b-instruct (revision c37e53c4bb): resid_post layer-19 trainers, k=64 and k=128 — fit: instruct-model SAE on instruct activations with LMSYS in its training mix; fraction of variance explained 0.8097 on our tokens vs 0.8057 published. Judge: claude-sonnet-4-5-20250929 (Anthropic Batch API). No WandB run (no training; fits log to JSON checkpoints). Feature-correlates round (0 GPU-h, VM-only): driver scripts/issue1482_feature_correlates.py; artifacts eval_results/issue_1482/feature_correlates (consistency.json, abstraction.json, consistency_perfeature.npz) + the two figures (PNG + PDF + data sidecars), all @ 9fbcb4dd8e; feature-level judge calls sync-routed (300 + 60 retest, max_tokens 400, 0 drops). Feature-extremes round (0 GPU-h, VM-only): driver scripts/issue1482_feature_extremes.py; artifacts eval_results/issue_1482/feature_extremes (extremes.json, extremes_perfeature.npz) + three figures (PNG + PDF + data sidecars) + the per-feature dashboard tasks/awaiting_promotion/1482/artifacts/feature_extremes_dashboard.html, all @ fcc9965459; judge sync-routed (358 + 60 retest, max_tokens 400, 0 content drops / 0 transport losses); extended rubric 2774598533dbdcc3, gate-verified as a verbatim-prefix additive extension of the reference rubric; r_B directions reused from the #779 monitoring bank (HF issue779_monitoring/r_b @ 037fcbb).

Context: Created 2026-07-17 from user chat; lineage: #779 — error-analysis child of the parent n1M map round. Origin prompt (verbatim): "Help me to setup and run this experiment: \subsection{What Is This Mapping Bad at Predicting?} - Analysis of worst predicted contexts: analysis/categorization of answers for which predictions are the worst. - Analysis of worst predicted SAE features [cunningham2023sparse, bricken2023monosemanticity, templeton2024scaling]: map from SAE features in the context to the answer and find the SAE features which are worst predicted by this. Use existing artifacts/trained mappings when possible. [second message:] can we also for the 1 million context mapping try to characterize which part is predictable linearly but not nonlinearly? and actually we can run all these experiments on the 1 million context mapping ideally". Plan v3 approved 2026-07-17; run 2026-07-18 (pod phases) through 2026-07-19 UTC (off-pod judge + analysis); interpretation passed critique and the body promoted in place 2026-07-19 UTC. Free-analysis follow-up round (2026-07-19, 0 GPU-h): the per-feature label-shuffle null, folded into Results the same day. Same-issue follow-up round k-resample-noise-floor (2026-07-19, ~3.1 GPU-h; scope: epm:followup-scope v1, source proposer-9b-cheap, est 4 GPU-h, question_relation same): K = 4 fresh answer draws per context measured the answer-sampling floor for the language arms; folded into Results 2026-07-19. Same-issue follow-up round perdirection-shrinkage-control (2026-07-19, ~1.3 GPU-h; scope: epm:followup-scope v2, source proposer-9b-cheap, est 3 GPU-h, question_relation same): per-direction ridge-λ tuning adjudicated the shrinkage caveat on the low-variance gap; folded into Results 2026-07-19. Same-issue inline free-analysis round feature-correlates (2026-07-27, 0 GPU-h, user-chat inline; origin prompt verbatim: "did we look at if high-level or low-level features were predicted better? Or look at it correlates with consistency of the SAE feature across the answer?" → "run both inline"): per-feature within-answer-consistency and judged-abstraction-level covariate reads over the committed per-feature R²; folded into Results 2026-07-27. Same-issue inline free-analysis round feature-extremes-dashboard (2026-07-28, 0 GPU-h, user-chat inline; origin prompt verbatim: "can you make a dashboard with the best predicted and worst predicted features, and then run the same experiment with the LLM judge inline on it", extended in-round with persona-relatedness + decoder-alignment reads): R²-tail selection (global + activity-decile-matched), extended-rubric judge, mechanical r_B alignment; folded into Results 2026-07-28. Same-issue follow-up round early-layer-arm (2026-07-28→29, ~4.6 realized GPU-h of 4 approved; scope: epm:followup-scope v3, source user-chat, est 4 GPU-h, question_relation same; origin prompt (verbatim): "add these to the existing experiments" — approving the taxonomy-review rank-1 gap "Early-layer (layers 0-4) token-level arm (~2-4 GPU-h)"): a layer-3 token-level arm at matched rows plus a fresh layer-19 control; judged feature-labelling frozen mid-round by user directive (verbatim: "can you pause those runs till the autointerp finishes?", plan v17 §0.-1) — the planned abstraction-level tests are deferred to #1773's validated labelling instrument, with the 377-feature evidence packet persisted for that resume; mechanical reads folded into Results 2026-07-29. Same-issue follow-up round matryoshka-tier (2026-07-29→30, ~5.6 realized GPU-h of 6 approved; scope: epm:followup-scope v4, source user-chat, est 6 GPU-h, question_relation same; origin prompt (verbatim): assistant proposed "encode a corpus-stratified subsample through BOTH matryoshka SAEs, fit context-features -> answer-features per dictionary, then report per-feature R2 by tier (within density decile), tier x judged level agreement, and the chat-vs-pile dictionary contrast (~4-6 GPU-h). Want me to queue it?" → user: "yes"): layer-20 matryoshka dictionaries supply a training-imposed granularity tier per feature as the judge-free analogue of the frozen abstraction-level labels; the tier × judged-level scope item stayed frozen per the standing labelling freeze (zero judge calls; the judged spec + 540 evidence packets persisted for the #1773 instrument's resume); mechanical + statistical reads folded into Results 2026-07-30.