On Distinguishing Capability Elicitation from Capability Creation in Post-Training: A Free-Energy Perspective

Authors: Yuhao Li, Shengchao Liu

arXiv · PDF

Summary

The authors argue that the real distinction in post-training isn't SFT versus RL but whether you're reweighting behaviors the model can already produce ("capability elicitation") versus expanding what it can practically reach ("capability creation"). They formalize this using "accessible support" — the set of behaviors a model can actually generate under realistic compute budgets — and show that both SFT and RL can be viewed through a free-energy lens where different signals (demonstrations or rewards) define what counts as "low energy." The key insight is that when updates stay close to the base model, you're mostly doing local reweighting, not creating new capabilities; capability creation requires search, interaction, or new information.

Main takeaways:

• Post-training should be analyzed by whether it reweights existing accessible behaviors (elicitation) or changes the reachable set (creation), not by whether it's labeled SFT or RL.
• "Accessible support" is the set of behaviors a model can practically produce under finite budgets; training that stays near the base model mainly reweights within this support.
• SFT and RL both reweight a pretrained reference distribution using external signals (demonstrations or rewards), so the method label is less important than the distance from the base model.
• Capability creation requires mechanisms like search, tool use, interaction, or incorporating genuinely new information, not just re-labeling existing training regimes.

Relevance

This connects directly to my installation-path equivalence work (prompt vs steering vs fine-tuning) — the accessible-support framework could explain when different installation methods are equivalent (all reweighting the same support) versus when one method genuinely expands what the model can reach, which might matter for marker implantation robustness.

Abstract

arXiv:2605.08368v1 Announce Type: new Abstract: Debates about large language model post-training often treat supervised fine-tuning (SFT) as imitation and reinforcement learning (RL) as discovery. But this distinction is too coarse. What matters is whether a training procedure increases the probability of behaviors the pretrained model could already produce, or whether it changes what the model can practically reach. We argue that post-training research should distinguish between capability elicitation and capability creation. We make this distinction operational by introducing the notion of accessible support: the set of behaviors that a model can practically produce under finite budgets. Post-training that reweights behaviors within this support is capability elicitation; whereas changing the support itself corresponds to capability creation. We develop this argument through a free-energy view of post-training. SFT and RL can both be seen as reweighting a pretrained reference distribution, only with different external signals. Demonstration signals define low-energy behavior for SFT, and reward signals define low-energy behavior for RL. When the update remains close to the base model, the main effect is local reweighting, not capability creation. Within this framework, the central question is no longer whether post-training is framed as SFT or RL, but whether it reweights behaviors already within reach, or instead expands the model's reachable behavioral space through search, interaction, tool use, or the incorporation of new information.