#2061 — SAE per-feature predictability change at each post-training stage
Generated 2026-08-07 14:20Z · commit 0a0a26fc14 · generator scripts/issue2061_predictability_dashboard.py · data: per-feature held-out R² (context/prefix → answer-state SAE features, EleutherAI/sae-llama-3.1-8b-64x L29, TopK k=32) from issue2061_sae_predictability/analysis_tensors/per_feature_r2 (HF), Tülu-3 ladder. ΔR²_j = R²_j(stage) − R²_j(previous stage), over features with a defined held-out R² at BOTH stages. Activation counts = rows where the feature appears in the TopK k=32 code with val ≠ 0 (per stage × corpus).
Read with #2061's headline caveats. (1) No per-feature gain survives the selection-symmetric max-statistic null (GLOBAL p97.5 = 1.82; largest observed ΔR² has p = 0.49) — every table below is DESCRIPTIVE, multiplicity-uncorrected. (2) The extreme ΔR² values are dominated by rarely-active features (activation counts are shown per row — treat 1–9-row features as noise-grade). (3) The fixed SAE dictionary explains only ~33% of answer-state variance, so per-feature reads are minority-variance-scoped. (4) Prefix arms on chat renders have a row-constant prefix input — at-chance by construction. (5) ⚠reg marks features whose ridge fit was regularization-limited at an endpoint (λ pinned at the audited grid edge; not a clean read.)
base → sft
context arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 150746 | +0.573 | 59 → 968 | 0.26 |
| chat/gsm8k_train_full | most decreased | 201259 | -0.303 | 181 → 66 | 0.26 |
| chat/if11k | most increased | 42742 | +1.057 | 1 → 4 | 1.77 |
| chat/if11k | most decreased | 242652 | -1.514 | 1 → 2 | 1.77 |
| chat/lmsys23k | most increased | 126412 | +0.761 | 2 → 6 | 0.61 |
| chat/lmsys23k | most decreased | 233043 | -0.567 | 319 → 8 | 0.61 |
| chat/math7500 | most increased | 169302 | +0.610 | 1486 → 3149 | 0.60 |
| chat/math7500 | most decreased | 90441 | -0.544 | 1155 → 2 | 0.60 |
| chat/sft11k | most increased | 165540 | +0.994 | 5 → 72 | 1.01 |
| chat/sft11k | most decreased | 234902 | -0.857 | 441 → 2 | 1.01 |
| chat/uf11k | most increased | 15545 | +0.918 | 4 → 9 | 1.11 |
| chat/uf11k | most decreased | 37725 | -0.937 | 5 → 2 | 1.11 |
| naturalistic/lmsys23k | most increased | 126412 | +0.802 | 2 → 6 | 0.54 |
| naturalistic/lmsys23k | most decreased | 186293 | -0.651 | 55 → 17 | 0.54 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 78077 | +0.938 | 2 | 0 → 1 | — | — | — | — | +0.965 | +0.912 | — |
| 120670 | +0.919 | 2 | 0 → 1 | — | — | — | — | +0.923 | +0.914 | — |
| 15545 | +0.916 | 2 | 0 → 1 | — | — | — | — | +0.914 | +0.918 | — |
| 121856 | +0.897 | 2 | 0 → 3 | — | — | — | — | +0.888 | +0.905 | — |
| 140806 | +0.776 | 4 | 2 → 4 | — | +0.941 | +0.746 | — | — | +0.693 | +0.725 |
| 43267 | +0.769 | 4 | 2 → 4 | — | +0.902 | +0.743 | — | — | +0.704 | +0.727 |
| 250030 | +0.745 | 4 | 1 → 4 | — | +0.875 | +0.703 | — | — | +0.732 | +0.671 |
| 28159 | +0.709 | 4 | 1 → 4 | — | +0.895 | +0.724 | — | — | +0.524 | +0.694 |
| 3535 | +0.703 | 5 | 3 → 4 | — | +0.895 | +0.747 | — | +0.524 | +0.592 | +0.757 |
| 42951 | +0.679 | 5 | 2 → 4 | — | +0.927 | +0.725 | — | +0.367 | +0.674 | +0.702 |
| 126412 | +0.668 | 4 | 1 → 3 | — | +0.616 | +0.761 | — | — | +0.491 | +0.802 |
| 259820 | +0.629 | 4 | 1 → 4 | — | +0.811 | +0.652 | — | — | +0.575 | +0.479 |
| 220076 | +0.623 | 5 | 13 → 44 | — | +0.791 | +0.544 | — | +0.612 | +0.570 | +0.596 |
| 179582 | +0.622 | 2 | 0 → 0 | — | — | +0.610 | — | — | — | +0.633 |
| 128129 | +0.619 | 5 | 11 → 42 | — | +0.723 | +0.716 | — | +0.652 | +0.263 | +0.742 |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 242652 | -0.554 | 3 | 0 → 1 | — | -1.514 | -0.075 | — | — | — | -0.074 |
| 86636 | -0.529 | 2 | 4 → 0 | — | — | — | — | -0.285 | -0.773 | — |
| 95505 | -0.508 | 2 | 5 → 0 | — | -0.939 | -0.078 | — | — | — | — |
| 234902 | -0.440 | 5 | 267 → 4 | — | -0.275 | -0.350 | — | -0.857 | -0.428 | -0.291 |
| 74958 | -0.433 | 4 | 153 → 1 | — | -0.383 | -0.549 | — | -0.409 | -0.392 | — |
| 246509 | -0.415 | 2 | 0 → 1 | — | — | -0.420 | — | — | — | -0.410 |
| 20488 | -0.414 | 2 | 0 → 0 | — | — | -0.566 | — | — | — | -0.261 |
| 389 | -0.411 | 2 | 0 → 0 | — | — | -0.562 | — | — | — | -0.260 |
| 144987 | -0.398 | 2 | 244 → 0 | — | — | -0.391 | — | — | — | -0.406 |
| 173853 | -0.396 | 5 | 240 → 1 | — | -0.474 | -0.446 | — | -0.391 | -0.470 | -0.198 |
| 52894 | -0.395 | 5 | 643 → 17 | — | -0.262 | -0.423 | — | -0.650 | -0.301 | -0.340 |
| 93779 | -0.387 | 3 | 0 → 1 | — | — | -0.416 | — | -0.323 | — | -0.422 |
| 14420 | -0.384 | 3 | 2 → 0 | — | -0.759 | — | — | -0.142 | -0.251 | — |
| 81065 | -0.382 | 2 | 1 → 0 | — | -0.522 | — | — | — | -0.243 | — |
| 216229 | -0.382 | 4 | 59 → 3 | — | -0.394 | -0.441 | — | — | -0.285 | -0.407 |
prefix arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 9054 | +0.087 ⚠reg | 1 → 71 | 0.04 |
| chat/gsm8k_train_full | most decreased | 106642 | -0.006 ⚠reg | 38 → 8 | 0.04 |
| chat/if11k | most increased | 110581 | +0.013 | 7 → 8 | 0.08 |
| chat/if11k | most decreased | 74958 | -0.023 | 633 → 10 | 0.08 |
| chat/lmsys23k | most increased | 62772 | +0.048 | 1 → 1 | 0.17 |
| chat/lmsys23k | most decreased | 10832 | -0.046 | 6414 → 30 | 0.17 |
| chat/math7500 | most increased | 117479 | +0.003 ⚠reg | 95 → 36 | 0.04 |
| chat/math7500 | most decreased | 220282 | -0.003 ⚠reg | 49 → 63 | 0.04 |
| chat/sft11k | most increased | 256891 | +0.010 | 1 → 2 | 0.09 |
| chat/sft11k | most decreased | 20496 | -0.022 | 5655 → 2623 | 0.09 |
| chat/uf11k | most increased | 63456 | +0.109 | 1 → 4 | 0.09 |
| chat/uf11k | most decreased | 20496 | -0.055 | 5857 → 3452 | 0.09 |
| naturalistic/lmsys23k | most increased | 175465 | +0.053 | 1 → 1 | 0.84 |
| naturalistic/lmsys23k | most decreased | 68412 | -0.135 | 1 → 1 | 0.84 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 20733 | +0.054 | 2 | 0 → 0 | — | — | — | — | — | +0.108 | +0.000 |
| 96264 | +0.052 | 2 | 0 → 0 | — | — | — | — | — | +0.108 | -0.004 |
| 22460 | +0.044 | 2 | 0 → 0 | — | — | — | — | +0.000 | +0.088 | — |
| 63456 | +0.036 | 3 | 0 → 3 | — | — | +0.000 | — | — | +0.109 | +0.000 |
| 197443 | +0.036 | 3 | 0 → 1 | — | — | +0.000 | — | — | +0.108 | +0.000 |
| 46337 | +0.036 | 3 | 0 → 1 | — | — | +0.001 | — | — | +0.108 | -0.002 |
| 221627 | +0.033 | 3 | 0 → 6 | — | — | +0.000 | — | — | +0.099 | +0.000 |
| 60039 | +0.029 | 3 | 0 → 1 | — | — | +0.000 | — | — | +0.087 | +0.000 |
| 75033 | +0.028 | 4 | 1 → 2 | — | — | +0.000 | — | +0.002 | +0.108 | +0.000 |
| 109375 | +0.027 | 4 | 1 → 3 | +0.000 | — | -0.001 | — | — | +0.108 | +0.001 |
| 175465 | +0.027 | 2 | 0 → 0 | — | — | +0.000 | — | — | — | +0.053 |
| 192872 | +0.027 | 4 | 2 → 1 | -0.000 | +0.000 | — | — | -0.000 | +0.107 | — |
| 70473 | +0.026 | 4 | 1 → 1 | — | +0.000 | +0.000 | — | — | +0.108 | -0.003 |
| 61748 | +0.025 | 2 | 0 → 2 | — | — | -0.000 | — | — | — | +0.051 |
| 134053 | +0.022 | 3 | 0 → 0 | — | +0.001 | — | — | +0.000 | +0.064 | — |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 160750 | -0.044 | 3 | 1 → 0 | — | — | — | — | +0.000 | +0.002 | -0.135 |
| 140460 | -0.032 | 4 | 596 → 1 | — | -0.020 | -0.030 | — | — | -0.028 | -0.049 |
| 20496 | -0.027 | 7 | 5964 → 4086 | -0.004 | -0.020 | -0.041 | +0.001 | -0.022 | -0.055 | -0.049 |
| 167452 | -0.026 | 2 | 251 → 0 | — | -0.017 | — | — | — | -0.034 | — |
| 74958 | -0.021 | 4 | 153 → 1 | — | -0.023 | -0.032 | — | -0.002 | -0.025 | — |
| 213051 | -0.021 | 5 | 612 → 5 | — | -0.011 | -0.022 | — | -0.001 | -0.024 | -0.045 |
| 10832 | -0.021 | 6 | 2739 → 7 | — | -0.022 | -0.046 | +0.002 | -0.014 | -0.042 | -0.001 |
| 38940 | -0.021 | 5 | 786 → 15 | — | -0.023 | -0.029 | — | -0.006 | -0.016 | -0.028 |
| 167925 | -0.020 | 5 | 949 → 81 | — | -0.017 | -0.020 | — | -0.003 | -0.021 | -0.037 |
| 55376 | -0.019 | 2 | 0 → 6 | — | — | -0.000 | — | — | — | -0.037 |
| 144987 | -0.019 | 2 | 244 → 0 | — | — | -0.014 | — | — | — | -0.023 |
| 128644 | -0.018 | 6 | 638 → 25 | +0.001 | -0.012 | -0.024 | — | -0.015 | -0.018 | -0.040 |
| 118829 | -0.018 | 5 | 965 → 13 | — | -0.017 | -0.041 | +0.001 | — | -0.031 | +0.001 |
| 25711 | -0.017 | 2 | 234 → 0 | — | — | -0.016 | — | — | — | -0.018 |
| 126755 | -0.017 | 2 | 0 → 0 | — | — | -0.006 | — | — | — | -0.027 |
sft → dpo
context arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 248415 | +0.335 | 1 → 31 | 0.34 |
| chat/gsm8k_train_full | most decreased | 237076 | -0.363 | 457 → 1 | 0.34 |
| chat/if11k | most increased | 45708 | +1.274 | 1 → 4 | 1.78 |
| chat/if11k | most decreased | 206836 | -1.553 | 3 → 1 | 1.78 |
| chat/lmsys23k | most increased | 251937 | +0.752 | 4 → 40 | 0.74 |
| chat/lmsys23k | most decreased | 219738 | -0.823 | 46 → 5 | 0.74 |
| chat/math7500 | most increased | 132181 | +0.488 | 104 → 1652 | 0.43 |
| chat/math7500 | most decreased | 14193 | -0.400 | 10 → 5 | 0.43 |
| chat/sft11k | most increased | 204677 | +1.098 | 7 → 7 | 1.10 |
| chat/sft11k | most decreased | 167817 | -1.060 | 3 → 1 | 1.10 |
| chat/uf11k | most increased | 137526 | +1.094 | 2 → 4 | 1.24 |
| chat/uf11k | most decreased | 100802 | -1.318 | 3 → 1 | 1.24 |
| naturalistic/lmsys23k | most increased | 251937 | +0.822 | 4 → 40 | 0.90 |
| naturalistic/lmsys23k | most decreased | 219738 | -0.889 | 48 → 6 | 0.90 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 214590 | +0.619 | 2 | 0 → 0 | — | — | +0.570 | — | — | — | +0.668 |
| 158363 | +0.618 | 2 | 0 → 0 | — | — | +0.569 | — | — | — | +0.667 |
| 226524 | +0.616 | 2 | 0 → 4 | — | — | — | — | +0.187 | +1.045 | — |
| 154238 | +0.615 | 2 | 0 → 0 | — | — | +0.569 | — | — | — | +0.662 |
| 22233 | +0.610 | 2 | 0 → 0 | — | — | +0.560 | — | — | — | +0.660 |
| 36681 | +0.588 | 2 | 1 → 0 | — | +0.314 | — | — | — | +0.862 | — |
| 190185 | +0.562 | 3 | 1 → 0 | — | +1.049 | +0.474 | — | — | — | +0.164 |
| 170195 | +0.527 | 2 | 0 → 0 | — | +0.068 | — | — | — | +0.986 | — |
| 227775 | +0.512 | 2 | 0 → 0 | — | — | — | — | +0.306 | +0.718 | — |
| 248819 | +0.503 | 3 | 0 → 1 | — | +0.210 | — | — | +0.562 | +0.739 | — |
| 40969 | +0.500 | 2 | 0 → 0 | — | +0.890 | — | — | — | +0.110 | — |
| 222180 | +0.496 | 3 | 0 → 0 | — | +0.639 | — | — | +0.263 | +0.587 | — |
| 118349 | +0.490 | 2 | 0 → 0 | — | — | +0.484 | — | — | — | +0.496 |
| 180974 | +0.489 | 2 | 0 → 0 | — | — | +0.483 | — | — | — | +0.496 |
| 193117 | +0.483 | 2 | 0 → 0 | — | +0.845 | — | — | +0.122 | — | — |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 170463 | -0.809 | 2 | 1 → 0 | — | — | -0.795 | — | — | — | -0.823 |
| 208204 | -0.700 | 2 | 2 → 0 | — | -1.532 | — | — | +0.132 | — | — |
| 81337 | -0.646 | 2 | 0 → 1 | — | -1.266 | — | — | — | — | -0.026 |
| 75171 | -0.639 | 3 | 7 → 0 | — | -0.702 | — | — | -0.721 | -0.494 | — |
| 54976 | -0.592 | 2 | 0 → 0 | — | -1.170 | — | — | -0.014 | — | — |
| 9559 | -0.584 | 2 | 0 → 0 | — | — | -0.550 | — | — | — | -0.619 |
| 211196 | -0.577 | 2 | 5 → 0 | — | -0.747 | — | — | -0.407 | — | — |
| 229403 | -0.563 | 2 | 0 → 0 | — | -0.745 | — | — | — | -0.382 | — |
| 114429 | -0.546 | 2 | 0 → 0 | — | -1.083 | — | — | — | -0.009 | — |
| 99764 | -0.541 | 2 | 1 → 0 | — | -1.071 | -0.012 | — | — | — | — |
| 206836 | -0.536 | 3 | 1 → 0 | — | -1.553 | -0.029 | — | — | — | -0.026 |
| 1919 | -0.529 | 2 | 2 → 0 | — | -0.944 | -0.114 | — | — | — | — |
| 74574 | -0.515 | 3 | 0 → 0 | — | -0.984 | — | — | -0.396 | -0.165 | — |
| 135341 | -0.499 | 3 | 6 → 0 | — | -0.895 | — | — | -0.154 | -0.448 | — |
| 84485 | -0.497 | 2 | 2 → 0 | — | — | — | — | -0.063 | -0.931 | — |
prefix arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 106642 | +0.006 ⚠reg | 8 → 1 | 0.55 |
| chat/gsm8k_train_full | most decreased | 259094 | -0.003 ⚠reg | 84 → 123 | 0.55 |
| chat/if11k | most increased | 246737 | +0.021 | 1 → 2 | 0.07 |
| chat/if11k | most decreased | 85924 | -0.034 | 11 → 6 | 0.07 |
| chat/lmsys23k | most increased | 251937 | +0.072 | 4 → 40 | 0.03 |
| chat/lmsys23k | most decreased | 30303 | -0.079 | 2 → 1 | 0.03 |
| chat/math7500 | most increased | 122648 | +0.003 ⚠reg | 6 → 2 | 0.03 |
| chat/math7500 | most decreased | 11307 | -0.004 ⚠reg | 13 → 43 | 0.03 |
| chat/sft11k | most increased | 61762 | +0.018 | 1 → 2 | 0.48 |
| chat/sft11k | most decreased | 128552 | -0.059 | 2 → 2 | 0.48 |
| chat/uf11k | most increased | 235701 | +0.030 | 4 → 2 | 0.16 |
| chat/uf11k | most decreased | 58061 | -0.049 | 2 → 3 | 0.16 |
| naturalistic/lmsys23k | most increased | 68412 | +0.135 | 1 → 2 | 0.67 |
| naturalistic/lmsys23k | most decreased | 61748 | -0.051 | 10 → 4 | 0.67 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 68412 | +0.068 | 2 | 0 → 0 | — | — | +0.000 | — | — | — | +0.135 |
| 249218 | +0.064 | 2 | 0 → 0 | — | — | +0.000 | — | — | — | +0.128 |
| 17258 | +0.064 | 2 | 0 → 0 | — | — | -0.000 | — | — | — | +0.128 |
| 251937 | +0.014 | 5 | 1 → 2 | — | +0.000 | +0.072 | — | +0.000 | +0.000 | -0.000 |
| 207303 | +0.013 | 5 | 4 → 5 | — | -0.000 | +0.067 | — | +0.001 | -0.000 | -0.000 |
| 36351 | +0.013 | 5 | 9 → 2 | — | -0.000 | +0.057 | — | -0.000 | +0.006 | +0.000 |
| 112697 | +0.012 | 3 | 1 → 0 | — | — | +0.006 | — | — | +0.000 | +0.029 |
| 200125 | +0.011 | 2 | 2 → 0 | — | — | +0.002 | — | — | — | +0.021 |
| 113500 | +0.011 | 4 | 4 → 1 | +0.000 | -0.000 | +0.046 | — | — | — | -0.001 |
| 95033 | +0.011 | 4 | 1 → 1 | — | +0.020 | -0.000 | — | — | +0.020 | +0.004 |
| 21215 | +0.011 | 5 | 7 → 2 | — | +0.001 | +0.048 | — | +0.005 | -0.000 | +0.000 |
| 2484 | +0.010 | 4 | 8 → 7 | — | — | +0.042 | — | +0.000 | -0.001 | +0.000 |
| 39982 | +0.010 | 2 | 1 → 0 | — | — | — | — | +0.000 | +0.021 | — |
| 253140 | +0.010 | 6 | 10 → 11 | — | +0.000 | +0.060 | +0.000 | -0.001 | -0.000 | -0.000 |
| 101825 | +0.010 | 3 | 0 → 0 | — | — | +0.000 | — | — | +0.000 | +0.029 |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 30303 | -0.040 | 2 | 0 → 0 | — | — | -0.079 | — | — | — | +0.000 |
| 113999 | -0.037 | 2 | 0 → 0 | — | — | -0.075 | — | — | — | +0.000 |
| 148504 | -0.031 | 2 | 0 → 0 | — | — | -0.062 | — | — | — | +0.000 |
| 149667 | -0.025 | 2 | 0 → 0 | — | — | — | — | -0.030 | -0.020 | — |
| 213363 | -0.021 | 3 | 1 → 0 | — | — | -0.062 | — | — | +0.000 | +0.000 |
| 167402 | -0.015 | 4 | 2 → 1 | — | +0.001 | -0.062 | — | -0.000 | +0.000 | — |
| 58061 | -0.015 | 4 | 2 → 1 | — | -0.000 | -0.012 | — | — | -0.049 | -0.000 |
| 241437 | -0.015 | 2 | 0 → 0 | — | -0.030 | — | — | — | +0.000 | — |
| 207993 | -0.015 | 2 | 0 → 0 | — | -0.030 | — | — | — | +0.000 | — |
| 261215 | -0.015 | 2 | 0 → 0 | — | — | — | — | -0.030 | +0.000 | — |
| 154379 | -0.015 | 2 | 1 → 0 | — | — | — | — | -0.029 | +0.000 | — |
| 223858 | -0.015 | 2 | 0 → 0 | — | — | — | — | -0.029 | +0.000 | — |
| 88746 | -0.015 | 2 | 0 → 0 | — | +0.000 | — | — | -0.030 | — | — |
| 249768 | -0.014 | 2 | 1 → 0 | — | -0.028 | — | — | — | +0.000 | — |
| 66054 | -0.014 | 2 | 0 → 0 | — | — | — | — | -0.028 | -0.000 | — |
dpo → rlvr
context arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 11031 | +0.172 | 1 → 2 | 0.21 |
| chat/gsm8k_train_full | most decreased | 141759 | -0.203 | 2 → 1 | 0.21 |
| chat/if11k | most increased | 175088 | +1.458 | 1 → 3 | 1.50 |
| chat/if11k | most decreased | 186369 | -1.206 | 3 → 1 | 1.50 |
| chat/lmsys23k | most increased | 219738 | +0.841 | 5 → 44 | 0.66 |
| chat/lmsys23k | most decreased | 251937 | -0.765 | 40 → 2 | 0.66 |
| chat/math7500 | most increased | 169765 | +0.297 | 2 → 8 | 0.58 |
| chat/math7500 | most decreased | 96101 | -0.387 | 5 → 4 | 0.58 |
| chat/sft11k | most increased | 220369 | +1.246 | 1 → 15 | 1.20 |
| chat/sft11k | most decreased | 164050 | -1.030 | 4 → 3 | 1.20 |
| chat/uf11k | most increased | 25293 | +1.013 | 2 → 2 | 1.36 |
| chat/uf11k | most decreased | 20140 | -1.174 | 2 → 1 | 1.36 |
| naturalistic/lmsys23k | most increased | 100405 | +0.974 | 3 → 40 | 0.81 |
| naturalistic/lmsys23k | most decreased | 251937 | -0.833 | 40 → 2 | 0.81 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 21956 | +0.659 | 2 | 0 → 0 | — | — | — | — | +0.725 | +0.592 | — |
| 70580 | +0.645 | 2 | 0 → 0 | — | — | +0.652 | — | — | — | +0.639 |
| 41534 | +0.588 | 2 | 0 → 0 | — | — | — | — | +0.982 | — | +0.195 |
| 3801 | +0.523 | 2 | 0 → 0 | — | +0.520 | — | — | — | +0.526 | — |
| 25293 | +0.502 | 2 | 1 → 0 | — | — | — | — | -0.010 | +1.013 | — |
| 206836 | +0.468 | 3 | 0 → 0 | — | +1.386 | -0.002 | — | — | — | +0.019 |
| 237270 | +0.463 | 2 | 0 → 0 | — | +0.511 | — | — | — | +0.415 | — |
| 40133 | +0.462 | 2 | 0 → 0 | — | — | +0.489 | — | — | — | +0.436 |
| 71334 | +0.453 | 2 | 0 → 0 | — | — | — | — | +0.256 | +0.650 | — |
| 101689 | +0.451 | 2 | 0 → 0 | — | +0.421 | — | — | — | +0.481 | — |
| 166638 | +0.443 | 2 | 0 → 0 | — | — | — | — | +0.109 | +0.776 | — |
| 210296 | +0.420 | 2 | 0 → 0 | — | — | +0.437 | — | — | — | +0.403 |
| 105107 | +0.418 | 2 | 0 → 0 | — | +0.375 | — | — | — | +0.462 | — |
| 122927 | +0.414 | 2 | 0 → 0 | — | +0.128 | — | — | — | +0.700 | — |
| 221745 | +0.387 | 2 | 0 → 0 | — | — | +0.373 | — | — | — | +0.402 |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 115039 | -0.738 | 2 | 0 → 0 | — | — | -0.664 | — | — | — | -0.813 |
| 201804 | -0.721 | 2 | 0 → 0 | — | — | -0.649 | — | — | — | -0.794 |
| 30242 | -0.660 | 2 | 0 → 0 | — | -0.997 | — | — | — | -0.323 | — |
| 4850 | -0.590 | 2 | 0 → 0 | — | — | -0.570 | — | — | — | -0.610 |
| 19653 | -0.574 | 2 | 0 → 0 | — | -0.438 | — | — | — | -0.709 | — |
| 258869 | -0.525 | 2 | 0 → 0 | — | -0.607 | — | — | — | -0.442 | — |
| 199550 | -0.518 | 2 | 1 → 0 | — | — | -0.512 | — | — | — | -0.524 |
| 163002 | -0.516 | 2 | 1 → 0 | — | — | -0.512 | — | — | — | -0.520 |
| 84863 | -0.514 | 2 | 0 → 0 | — | -0.982 | — | — | — | -0.046 | — |
| 122844 | -0.513 | 2 | 1 → 0 | — | — | — | — | -0.028 | -0.998 | — |
| 36681 | -0.499 | 2 | 0 → 0 | — | -0.192 | — | — | — | -0.806 | — |
| 231658 | -0.473 | 2 | 0 → 0 | — | — | -0.542 | — | — | — | -0.404 |
| 129425 | -0.463 | 2 | 0 → 0 | — | -0.895 | — | — | -0.030 | — | — |
| 186901 | -0.459 | 2 | 0 → 0 | — | — | -0.516 | — | — | — | -0.402 |
| 33489 | -0.458 | 2 | 0 → 0 | — | — | -0.483 | — | — | — | -0.434 |
prefix arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 226366 | +0.003 ⚠reg | 7 → 5 | 0.02 |
| chat/gsm8k_train_full | most decreased | 18370 | -0.003 ⚠reg | 26 → 53 | 0.02 |
| chat/if11k | most increased | 85924 | +0.035 | 6 → 6 | 0.13 |
| chat/if11k | most decreased | 98572 | -0.044 | 2 → 3 | 0.13 |
| chat/lmsys23k | most increased | 30303 | +0.079 | 1 → 1 | 0.06 |
| chat/lmsys23k | most decreased | 251937 | -0.071 | 40 → 2 | 0.06 |
| chat/math7500 | most increased | 35273 | +0.003 ⚠reg | 315 → 251 | 0.04 |
| chat/math7500 | most decreased | 152554 | -0.003 ⚠reg | 3 → 4 | 0.04 |
| chat/sft11k | most increased | 128552 | +0.047 | 2 → 3 | 0.52 |
| chat/sft11k | most decreased | 111576 | -0.024 | 1 → 1 | 0.52 |
| chat/uf11k | most increased | 58061 | +0.049 | 3 → 2 | 0.14 |
| chat/uf11k | most decreased | 122918 | -0.032 | 3 → 4 | 0.14 |
| naturalistic/lmsys23k | most increased | 64203 | +0.019 | 7 → 4 | 0.04 |
| naturalistic/lmsys23k | most decreased | 247960 | -0.020 | 9 → 14 | 0.04 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 148504 | +0.031 | 2 | 0 → 0 | — | — | +0.062 | — | — | — | -0.000 |
| 221716 | +0.031 | 2 | 0 → 0 | — | -0.000 | +0.062 | — | — | — | — |
| 30303 | +0.026 | 3 | 0 → 1 | — | — | +0.079 | — | — | +0.000 | -0.000 |
| 186518 | +0.024 | 2 | 0 → 1 | — | +0.023 | — | — | — | +0.024 | — |
| 58061 | +0.020 | 3 | 1 → 0 | — | -0.001 | +0.012 | — | — | +0.049 | — |
| 152038 | +0.016 | 2 | 0 → 0 | — | +0.032 | — | — | — | -0.000 | — |
| 244396 | +0.016 | 2 | 0 → 1 | — | -0.000 | — | — | +0.032 | — | — |
| 18162 | +0.016 | 4 | 1 → 2 | — | — | +0.062 | — | +0.000 | -0.000 | +0.000 |
| 241437 | +0.015 | 2 | 0 → 0 | — | +0.030 | — | — | — | +0.000 | — |
| 261215 | +0.015 | 2 | 0 → 0 | — | — | — | — | +0.030 | +0.000 | — |
| 101294 | +0.015 | 2 | 0 → 0 | — | — | — | — | +0.030 | -0.000 | — |
| 207993 | +0.015 | 2 | 0 → 0 | — | +0.030 | — | — | — | -0.000 | — |
| 59522 | +0.015 | 2 | 0 → 0 | — | — | — | — | +0.029 | +0.000 | — |
| 223858 | +0.015 | 2 | 0 → 0 | — | — | — | — | +0.029 | -0.000 | — |
| 154379 | +0.015 | 2 | 0 → 0 | — | — | — | — | +0.029 | -0.000 | — |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 98572 | -0.022 | 2 | 0 → 0 | — | -0.044 | — | — | -0.000 | — | — |
| 35000 | -0.019 | 2 | 0 → 0 | — | — | -0.022 | — | — | — | -0.017 |
| 251937 | -0.018 | 4 | 2 → 1 | — | — | -0.071 | — | -0.000 | -0.000 | +0.000 |
| 111127 | -0.016 | 2 | 1 → 0 | — | -0.032 | — | — | — | -0.000 | — |
| 50105 | -0.016 | 3 | 1 → 0 | — | -0.028 | — | — | -0.020 | +0.000 | — |
| 113500 | -0.016 | 3 | 1 → 0 | — | +0.001 | -0.046 | — | — | — | -0.003 |
| 174008 | -0.015 | 2 | 0 → 1 | — | -0.030 | — | — | +0.000 | — | — |
| 253580 | -0.014 | 2 | 1 → 0 | — | — | -0.012 | — | — | -0.017 | — |
| 36351 | -0.014 | 4 | 2 → 3 | — | -0.001 | -0.057 | — | -0.000 | — | +0.000 |
| 187400 | -0.014 | 2 | 0 → 0 | — | -0.029 | — | — | — | +0.000 | — |
| 207303 | -0.014 | 5 | 5 → 6 | — | -0.001 | -0.068 | — | -0.001 | -0.000 | +0.000 |
| 220375 | -0.012 | 2 | 0 → 0 | — | — | -0.015 | — | — | — | -0.010 |
| 156699 | -0.012 | 2 | 0 → 0 | — | — | -0.015 | — | — | — | -0.010 |
| 139510 | -0.012 | 2 | 0 → 0 | — | — | -0.015 | — | — | — | -0.010 |
| 27903 | -0.012 | 2 | 0 → 0 | — | — | -0.015 | — | — | — | -0.010 |
rlvr → longer-rlvr
context arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 69757 | +0.280 | 536 → 1242 | 0.27 |
| chat/gsm8k_train_full | most decreased | 96013 | -0.308 | 1021 → 71 | 0.27 |
| chat/if11k | most increased | 100309 | +1.050 | 6 → 402 | 1.34 |
| chat/if11k | most decreased | 197351 | -1.204 | 3 → 1 | 1.34 |
| chat/lmsys23k | most increased | 116382 | +0.949 | 1 → 9 | 0.94 |
| chat/lmsys23k | most decreased | 219738 | -0.819 | 44 → 5 | 0.94 |
| chat/math7500 | most increased | 214687 | +0.526 | 2 → 15 | 0.47 |
| chat/math7500 | most decreased | 128974 | -0.397 | 3 → 2 | 0.47 |
| chat/sft11k | most increased | 164050 | +0.998 | 3 → 4 | 1.13 |
| chat/sft11k | most decreased | 220369 | -1.218 | 15 → 1 | 1.13 |
| chat/uf11k | most increased | 66488 | +1.270 | 1 → 75 | 1.16 |
| chat/uf11k | most decreased | 132509 | -1.031 | 2 → 1 | 1.16 |
| naturalistic/lmsys23k | most increased | 144453 | +0.953 | 1 → 10 | 0.95 |
| naturalistic/lmsys23k | most decreased | 100405 | -0.874 | 40 → 3 | 0.95 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 121406 | +0.987 | 2 | 0 → 0 | — | +1.022 | — | — | — | +0.953 | — |
| 12354 | +0.929 | 2 | 0 → 0 | — | — | +0.926 | — | — | — | +0.932 |
| 213679 | +0.919 | 2 | 0 → 2 | — | +0.827 | — | — | — | +1.012 | — |
| 13809 | +0.858 | 2 | 0 → 0 | — | — | +0.828 | — | — | — | +0.888 |
| 115102 | +0.826 | 2 | 0 → 0 | — | +0.880 | — | — | — | +0.771 | — |
| 249795 | +0.792 | 2 | 1 → 0 | — | +0.895 | — | — | — | +0.688 | — |
| 25301 | +0.775 | 2 | 0 → 1 | — | +0.796 | — | — | — | +0.753 | — |
| 115039 | +0.763 | 2 | 0 → 0 | — | — | +0.706 | — | — | — | +0.820 |
| 123569 | +0.752 | 2 | 0 → 0 | — | +0.912 | — | — | — | +0.592 | — |
| 190248 | +0.736 | 2 | 0 → 0 | — | +0.957 | — | — | — | +0.515 | — |
| 117211 | +0.730 | 2 | 0 → 0 | — | — | +0.683 | — | — | — | +0.777 |
| 3644 | +0.712 | 2 | 1 → 0 | — | +0.844 | — | — | — | +0.580 | — |
| 44612 | +0.683 | 2 | 0 → 0 | — | +0.590 | — | — | — | +0.775 | — |
| 20909 | +0.654 | 2 | 0 → 0 | — | +0.203 | — | — | — | +1.104 | — |
| 205657 | +0.593 | 3 | 0 → 1 | — | +0.810 | — | — | +0.028 | +0.942 | — |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 44057 | -0.815 | 3 | 0 → 0 | — | -1.115 | — | — | -0.584 | -0.747 | — |
| 47203 | -0.789 | 2 | 0 → 0 | — | -0.979 | — | — | — | -0.600 | — |
| 70580 | -0.683 | 2 | 0 → 0 | — | — | -0.701 | — | — | — | -0.665 |
| 126212 | -0.651 | 2 | 0 → 0 | — | — | — | — | -0.796 | -0.506 | — |
| 41534 | -0.609 | 2 | 0 → 1 | — | — | — | — | -0.990 | — | -0.227 |
| 110815 | -0.581 | 2 | 0 → 0 | — | — | — | — | -0.825 | -0.337 | — |
| 204332 | -0.580 | 2 | 0 → 0 | — | -1.134 | — | — | — | -0.026 | — |
| 238117 | -0.580 | 2 | 0 → 1 | — | -0.203 | — | — | — | -0.956 | — |
| 10996 | -0.574 | 2 | 0 → 0 | — | — | — | — | -0.591 | -0.558 | — |
| 180120 | -0.567 | 2 | 0 → 0 | — | — | -0.490 | — | — | — | -0.644 |
| 247464 | -0.558 | 2 | 0 → 0 | — | — | — | — | -0.581 | -0.535 | — |
| 104910 | -0.479 | 2 | 0 → 0 | — | -0.584 | — | — | -0.373 | — | — |
| 118349 | -0.476 | 2 | 0 → 0 | — | — | -0.459 | — | — | — | -0.492 |
| 227295 | -0.475 | 2 | 0 → 0 | — | — | -0.458 | — | — | — | -0.491 |
| 45923 | -0.474 | 2 | 0 → 0 | — | — | -0.456 | — | — | — | -0.492 |
prefix arm
Per-cell winners (each render/corpus combo)
| render/corpus | winner | feature id | ΔR² | rows active (before → after) | cell max-null p97.5 |
|---|
| chat/gsm8k_train_full | most increased | 129058 | +0.003 ⚠reg | 31 → 116 | 0.03 |
| chat/gsm8k_train_full | most decreased | 16375 | -0.004 ⚠reg | 19 → 31 | 0.03 |
| chat/if11k | most increased | 98572 | +0.044 | 3 → 3 | 0.24 |
| chat/if11k | most decreased | 97790 | -0.021 | 2 → 1 | 0.24 |
| chat/lmsys23k | most increased | 35000 | +0.022 | 2 → 4 | 0.05 |
| chat/lmsys23k | most decreased | 21215 | -0.034 | 13 → 17 | 0.05 |
| chat/math7500 | most increased | 6517 | +0.003 ⚠reg | 12 → 10 | 0.03 |
| chat/math7500 | most decreased | 243531 | -0.003 ⚠reg | 100 → 78 | 0.03 |
| chat/sft11k | most increased | 36400 | +0.025 | 2 → 2 | 0.64 |
| chat/sft11k | most decreased | 11158 | -0.067 | 2 → 1 | 0.64 |
| chat/uf11k | most increased | 122918 | +0.032 | 4 → 1 | 0.07 |
| chat/uf11k | most decreased | 20496 | -0.049 | 1942 → 2307 | 0.07 |
| naturalistic/lmsys23k | most increased | 200713 | +0.023 | 2 → 5 | 0.04 |
| naturalistic/lmsys23k | most decreased | 73355 | -0.027 | 4 → 3 | 0.04 |
Most increased — top 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 50105 | +0.024 | 2 | 0 → 0 | — | +0.028 | — | — | +0.020 | — | — |
| 35000 | +0.019 | 2 | 0 → 0 | — | — | +0.022 | — | — | — | +0.017 |
| 255562 | +0.016 | 2 | 1 → 0 | — | +0.032 | -0.000 | — | — | — | — |
| 45193 | +0.015 | 2 | 0 → 0 | — | +0.030 | — | — | — | -0.000 | — |
| 164046 | +0.015 | 2 | 0 → 0 | — | — | — | — | — | +0.019 | +0.010 |
| 44278 | +0.014 | 2 | 0 → 1 | — | +0.028 | — | — | -0.000 | — | — |
| 197922 | +0.013 | 2 | 1 → 0 | — | +0.026 | — | — | -0.000 | — | — |
| 100151 | +0.013 | 2 | 1 → 0 | — | — | +0.011 | — | +0.014 | — | — |
| 27903 | +0.012 | 2 | 0 → 0 | — | — | +0.015 | — | — | — | +0.010 |
| 214000 | +0.012 | 2 | 0 → 0 | — | — | +0.015 | — | — | — | +0.010 |
| 219463 | +0.012 | 2 | 0 → 0 | — | — | +0.015 | — | — | — | +0.010 |
| 62693 | +0.012 | 2 | 0 → 0 | — | — | +0.015 | — | — | — | +0.010 |
| 246608 | +0.012 | 2 | 0 → 0 | — | — | +0.015 | — | — | — | +0.010 |
| 139510 | +0.012 | 2 | 0 → 0 | — | — | +0.015 | — | — | — | +0.010 |
| 156699 | +0.012 | 2 | 0 → 0 | — | — | +0.015 | — | — | — | +0.010 |
Most decreased — bottom 15 by mean ΔR² across combos (≥2 combos defined)
| feature id | mean ΔR² | #combos defined | median rows active (before → after) | chat/gsm8k_train_full | chat/if11k | chat/lmsys23k | chat/math7500 | chat/sft11k | chat/uf11k | naturalistic/lmsys23k |
|---|
| 111996 | -0.031 | 2 | 0 → 0 | -0.001 | — | — | — | -0.061 | — | — |
| 11158 | -0.022 | 3 | 0 → 0 | — | -0.000 | — | — | -0.067 | -0.000 | — |
| 255756 | -0.021 | 2 | 0 → 0 | — | — | -0.026 | — | — | — | -0.016 |
| 57850 | -0.021 | 2 | 0 → 0 | — | — | -0.026 | — | — | — | -0.016 |
| 97380 | -0.020 | 2 | 0 → 0 | — | — | -0.000 | — | -0.041 | — | — |
| 68204 | -0.020 | 2 | 1 → 0 | — | -0.000 | — | — | -0.041 | — | — |
| 260049 | -0.020 | 2 | 0 → 0 | — | +0.000 | — | — | -0.041 | — | — |
| 151657 | -0.020 | 2 | 0 → 0 | — | — | — | — | -0.040 | +0.000 | — |
| 20496 | -0.019 | 6 | 1942 → 3186 | -0.001 | -0.006 | -0.016 | — | -0.029 | -0.049 | -0.012 |
| 255677 | -0.018 | 2 | 0 → 0 | — | -0.000 | — | — | -0.037 | — | — |
| 239059 | -0.018 | 2 | 0 → 0 | — | — | -0.025 | — | — | — | -0.011 |
| 77557 | -0.018 | 2 | 0 → 0 | — | — | -0.020 | — | — | — | -0.016 |
| 228529 | -0.018 | 2 | 0 → 0 | — | — | -0.020 | — | — | — | -0.016 |
| 112616 | -0.016 | 2 | 0 → 0 | — | — | — | — | -0.033 | — | -0.000 |
| 13790 | -0.016 | 2 | 0 → 0 | — | — | — | — | -0.032 | -0.000 | — |
Feature ids index EleutherAI/sae-llama-3.1-8b-64x layer 29; no autointerp labels are available in-repo. Task: #2061.