#2061 — SAE feature activation frequency vs per-feature ΔR²
Each point is one SAE feature with a defined held-out R² at BOTH stages of the
selected transition (map: pooled context/prefix activation → answer-state SAE feature vector,
EleutherAI/sae-llama-3.1-8b-64x L29, TopK k=32, Tülu-3 ladder). x = rows where the feature is
active in the TopK k=32 code (min over the two stages; log axis, 0 plotted at 0.6, ±5%% x-jitter
for visibility — hover shows exact counts). y = ΔR² (later − earlier stage). Hover shows the
feature's auto-interp description (EleutherAI/auto_interp_interpretations).
Read with #2061's headline caveats. (1) No per-feature gain survives the selection-symmetric max-statistic null (GLOBAL p97.5 = 1.82) — every point below is DESCRIPTIVE, multiplicity-uncorrected. (2) Extreme ΔR² values are dominated by rarely-active features — treat points at the left edge (1–9 rows active) as noise-grade; the frequency axis exists to make exactly that visible. (3) The fixed SAE dictionary explains only ~33% of answer-state variance, so per-feature reads are minority-variance-scoped. (4) Prefix arms on chat renders have a row-constant prefix input — at-chance by construction. (5) Auto-interp descriptions cover 156,110/262,144 features (EleutherAI, generated on base-model token-level activations — they describe the base dictionary, not any post-trained stage) and are themselves LLM-generated: treat as hints, not ground truth. (6) ⚠reg marks features whose ridge fit was regularization-limited at an endpoint (λ pinned at the audited grid edge; not a clean read).