Reasoning emerges from constrained inference manifolds in large language models

Authors: Yanbiao Ma, Fei Luo, Linfeng Zhang et al.

arXiv · PDF

Summary

The authors examine what happens inside language models during reasoning, not just whether they get the right answer. They find that the internal representations during inference compress into low-dimensional manifolds (the model's internal state lives in a much smaller space than you'd expect). Good reasoning only happens when three conditions hold: the compressed space is expressive enough, compression happens spontaneously, and information doesn't collapse to zero in the compressed subspace. When these conditions fail, you get predictable failure modes. They propose a diagnostic that doesn't need any labels—just watch the internal dynamics.

Main takeaways:

• Reasoning compresses model representations into low-dimensional manifolds, but compression alone doesn't guarantee good reasoning
• Three necessary conditions: adequate expressivity, spontaneous compression, and preserved information volume in the compressed subspace
• Models outside this "structural regime" show characteristic pathological inference patterns
• The authors built a label-free diagnostic computed purely from internal dynamics, offering an alternative to benchmark evaluation
• Suggests reasoning is fundamentally governed by geometric and informational constraints, not just learned patterns

Relevance

Directly relevant to my persona and conditional behavior work—if personas are "compressed subspaces" or manifolds in representation space, this paper's geometric framework might explain why certain personas are stable vs. unstable, and could inform my cosine-distance persona geometry findings (#340, #237).

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses benchmark.

Abstract

arXiv:2605.08142v1 Announce Type: new Abstract: Reasoning in large language models is predominantly evaluated through labeled benchmarks, conflating task performance with the quality of internal inference. Here we study reasoning as an intrinsic dynamical process by examining the evolution of internal representations during inference. We find that inference-time dynamics consistently self-organize into low-dimensional manifolds embedded within high-dimensional representation spaces. we find that such geometric compression, although pervasive, is not sufficient for stable or reliable reasoning. Instead, effective reasoning dynamics emerge within a constrained structural regime characterized by three conditions: adequate representational expressivity, spontaneous manifold compression, and preservation of non-degenerate information volume within the compressed subspace. Models outside this regime exhibit characteristic pathological inference dynamics. Based on these insights, we introduce a unified, label-free diagnostic computed solely from internal dynamics. These findings suggest that reasoning in LLMs is fundamentally governed by geometric and informational constraints, offering a complementary framework to benchmark-centric assessment.