LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models
Authors: Taekhyun Park, Yongjae Lee, Dohee Kim et al.
Summary
The authors introduce LoopUS (Looped Depth Up-Scaling), a post-training method that converts a standard pretrained LLM into a looped architecture that iteratively refines hidden representations without generating longer output sequences. Instead of training a recurrent model from scratch or doing major architectural retrofits, LoopUS decomposes the pretrained model into an encoder, a looped reasoning block, and a decoder, using techniques like input-dependent selective gates (to prevent hidden-state drift), random deep supervision (for memory-efficient training over long loops), and a confidence head for adaptive early stopping. This lets you scale test-time compute through latent iteration while preserving pretrained capabilities.
Main takeaways:
- LoopUS converts a pretrained LLM into a looped architecture post-training, enabling iterative latent refinement without extending generated sequences
- Architecture: encoder → looped reasoning block → decoder, with selective gates to prevent drift and random deep supervision for efficient training
- Adaptive early exit via a confidence head allows variable compute based on problem difficulty
- Improves reasoning performance through test-time compute scaling without recurrent training from scratch or disrupting pretrained knowledge
Relevance
Tangentially related—this is about test-time compute scaling through looped inference, not persona installation. Relevant only if I ever explored whether repeated internal processing (without prompt extension) could amplify or stabilize persona markers, but the connection is loose. The "post-training recasting" is vaguely similar in spirit to my interest in behavioral installation paths.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses limitation, limitations.
Abstract
arXiv:2605.11011v1 Announce Type: new Abstract: Looped computation shows promise in improving the reasoning-oriented performance of LLMs by scaling test-time compute. However, existing approaches typically require either training recurrent models from scratch or applying disruptive retrofits, which involve substantial computational costs and may compromise pretrained capabilities. To address these limitations, we introduce \textbf{Looped Depth Up-Scaling} (LoopUS), a post-training framework that converts a standard pretrained LLM into a looped architecture. As a key technical contribution, LoopUS recasts the pretrained LLM into an encoder, a looped reasoning block, and a decoder. It operationalizes this latent-refinement architecture through four core components: (1) block decomposition, guided by staged representation dynamics; (2) an input-dependent selective gate to mitigate hidden-state drift; (3) random deep supervision for memory-efficient learning over long recursive horizons; and (4) a confidence head for adaptive early exiting. Collectively, these mechanisms transform a standard non-looped model into a looped form while stabilizing it against both computational bottlenecks and representation collapse. Through stable latent looping, LoopUS improves reasoning-oriented performance without extending the generated traces or requiring recurrent training from scratch. For more details, see https://thrillcrazyer.github.io/LoopUS