Structural Rationale Distillation via Reasoning Space Compression

Authors: Jialin Yang, Jiankun Wang, Jiajun Wu et al.

arXiv · PDF

Summary

When distilling reasoning from a large teacher model into a smaller student, the teacher's step-by-step explanations for similar problems often vary wildly, creating noisy supervision. The authors propose D-RPC, which maintains a compact bank of reusable high-level reasoning "paths" (templates) and forces the teacher to follow the most relevant one for each question, producing more consistent rationales. A PAC-Bayes analysis shows an optimal intermediate bank size: too small misses coverage, too large adds noise. Across five reasoning benchmarks, D-RPC beats standard chain-of-thought distillation and other baselines while using fewer tokens.

Main takeaways:

• Teacher models generate inconsistent reasoning paths for similar problems, which hurts student learning during distillation
• Constraining the teacher to follow retrieved high-level reasoning templates improves student performance
• There's a Goldilocks bank size: too few templates miss problem types, too many add supervision noise
• D-RPC outperforms chain-of-thought distillation and other methods on math and commonsense benchmarks with lower token cost

Relevance

Tangential but conceptually interesting for my work on installation-path equivalence—this shows that constraining the reasoning path (like using a template) can be more effective than freeform generation, which parallels my question about whether prompts, steering, and fine-tuning install behaviors through equivalent or different paths.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses benchmark.

Abstract

arXiv:2605.07139v1 Announce Type: new Abstract: When distilling reasoning from large language models (LLMs) into smaller ones, teacher rationales for similar problems often vary wildly in structure and strategy. Like a chef who makes the same dish differently each time, this inconsistency burdens the student with noisy supervision that is hard to internalize. We propose Distillation through Reasoning Path Compression (D-RPC), which constrains the teacher to follow a compact, dynamically maintained bank of reusable high-level reasoning paths. For each training question, D-RPC retrieves the most relevant path and conditions the teacher to follow it, producing rationales that are consistent across similar problems yet diverse enough to cover different problem types. A PAC-Bayes analysis formalizes the resulting trade-off between bank size and coverage: smaller banks reduce supervision entropy but risk coverage gaps, and the generalization bound identifies an optimal intermediate size confirmed by our ablations. Across five math and commonsense reasoning benchmarks with two student models, D-RPC consistently outperforms chain-of-thought distillation, freeform rationale generation, direct distillation, and structured-supervision baselines, while using fewer tokens than template-heavy alternatives.