EPS Dashboard

Authors: Yongzhong Xu

Summary

The paper empirically tests a theoretical "feature repulsion" mechanism from Tian (2025) that predicts similar learned features should repel each other during the grokking phase (delayed generalization) of two-layer network training. The authors confirm the predicted sign structure holds robustly across modular addition tasks, but find that the spectral signature in weight updates—whether repulsion shows up as a rank-2 eigenvalue gap—depends entirely on the activation function. With squared activations (x²), a clear rank-2 spectrum emerges; with ReLU, the spectrum stays rank-1 and repulsion is invisible in the weight updates, even though the sign structure is still correct.

Main takeaways:

Tian's feature-repulsion sign rule (similar features have negative off-diagonal entries in the B matrix) holds empirically with high accuracy in grokking setups.
The spectral signature of repulsion—a detectable rank-2 eigenvalue gap in weight updates—only appears with x² activation, not ReLU.
With x², a simple eigengap detector fires reliably at the grokking transition (epoch ~174) in all 15 seeds, and never in non-grokking controls.
With ReLU, the detector never fires and the spectrum remains rank-1, despite identical sign-structure validity.
This activation-dependent dissociation aligns with Tian's distinction between "focused" (power-law) and "spreading" (ReLU) memorization dynamics.

Relevance

Not directly related to persona or midtraining work—this is about grokking dynamics in toy neural networks. Tangentially interesting because it's about when learned structure becomes empirically observable in training dynamics, which loosely parallels my interest in when behavioral markers or personas become detectable during fine-tuning.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses negative.

Abstract

arXiv:2605.08119v1 Announce Type: new Abstract: Tian (2025) proves a repulsion theorem (Theorem 6) for the matrix $ B = (\widetilde{F}^\top \widetilde{F} + \eta I)^{-1} $ during the interactive feature-learning stage of grokking: similar features have negative off-diagonal entries $ B_{j\ell} $, producing an effective repulsive force that drives them apart. However, the theorem does not specify when this mechanism becomes empirically observable, nor whether it leaves a measurable spectral signature in the parameter updates. We test this directly on Tian's modular addition setup ($ M = 71 $, $ K = 2048 $, MSE loss) and observe a clear structure-mechanism dissociation. The predicted sign rule holds robustly on the top-200 most-similar feature pairs across activations (empirical sign-match rising from 0.865 to 0.985 on $ \sigma = x^2 $ across 5 seeds, and saturating at 1.000 on $ \sigma = \operatorname{ReLU} $). However, the spectral signature in the parameter updates is strongly activation-dependent. With $ \sigma = x^2 $, a simple slope detector on the rolling eigengap $ \sigma_2 / \sigma_3 $ of $ \Delta W $ fires in 15/15 grokking seeds at epoch 174 (IQR [173,174]) and in 0/15 non-grokking controls, with 229$ \times $ late-stage magnitude separation; the spectrum is rank-2. In contrast, with $ \sigma = \operatorname{ReLU} $, the detector never fires and the spectrum remains effectively rank-1. This dissociation aligns with Tian's Theorem 5 distinction between focused (power-law) and spreading (ReLU) memorization: while the sign structure of $ B $ depends only on $ \widetilde{F}^\top \widetilde{F} $, how feature repulsion translates into weight updates critically depends on the activation derivative $ \sigma' $.