Efficient LLM Reasoning via Variational Posterior Guidance with Efficiency Awareness
Authors: Zizhao Chen, Yuying Li, Siting Lin et al.
Summary
The authors tackle the "overthinking" problem in LLM reasoning—when chain-of-thought gets inefficiently long—by treating efficient reasoning as a variational inference problem. They introduce VPG-EA (Variational Posterior Guidance with Efficiency Awareness), which uses a dual-stream architecture where one stream learns a posterior distribution conditioned on reference answers (which can sample high-quality reasoning paths more easily) and the other is the prior policy used at inference. After filtering pseudo-efficient paths via cross-evaluation, they distill the posterior's efficient reasoning patterns into the prior policy. On DeepSeek-R1-Distill models (1.5B and 7B), this improves a composite efficiency metric (epsilon cubed) by 8.73% and 12.37% over baselines.
Main takeaways:
- Framing efficient reasoning as variational inference: a posterior distribution conditioned on correct answers can sample high-quality reasoning chains more easily than the prior policy, breaking the sampling bottleneck
- VPG-EA uses a parameter-shared dual-stream architecture to learn both posterior and prior, then distills efficient reasoning patterns from posterior to prior
- Cross-view evaluation filters out "pseudo-efficient" paths before distillation to ensure quality
- Improves composite efficiency metric by ~9-12% over strongest baselines on DeepSeek-R1-Distill-Qwen models
Relevance
Tangentially related—this is about compressing reasoning chains, not persona behavior, but the dual-stream architecture (posterior with answer access vs. prior without) loosely echoes my interest in different paths to the same behavior (fine-tuning vs. prompting). The "distilling one distribution into another" setup might be relevant if I ever explored training-time vs. inference-time persona installation.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses evaluation.
Abstract
arXiv:2605.11019v1 Announce Type: new Abstract: Although large language models rely on chain-of-thought for complex reasoning, the overthinking phenomenon severely degrades inference efficiency. Existing reinforcement learning methods compress reasoning chains by designing elaborate reward functions, which renders high-quality samples extremely sparse in the exploration space and creates a sampling bottleneck for the prior policy. Inspired by cognitive science, we theoretically prove that a posterior distribution guided by reference answers achieves higher expected utility than the prior distribution, thus capable of breaking through the sampling bottleneck of high-quality samples. However, the posterior distribution is unavailable during inference. To this end, we formalize efficient reasoning as a variational inference problem and introduce an efficiency-aware evidence lower bound as the theoretical foundation. Based on this, we propose the VPG-EA framework. It adopts a parameter-shared dual-stream architecture to instantiate both the posterior distribution and the prior policy; after filtering out pseudo-efficient paths via cross-view evaluation, it unidirectionally transfers the posterior's efficient patterns to the prior policy through variational distillation. Experiments on DeepSeek-R1-Distill-Qwen-1.5B and 7B scales demonstrate that VPG-EA improves the comprehensive efficiency metric epsilon cubed by 8.73% and 12.37% over the strongest baselines on each model size, respectively.