StoicLLM: Preference Optimization for Philosophical Alignment in Small Language Models
Authors: Ishmam Khan, Sindhuja Thogarrati, Shuo Zhang
Summary
The authors fine-tune small language models on micro-datasets of Stoic philosophy texts (just 300 examples) using preference optimization methods (ORPO, AlphaPO) to see if models can internalize a nuanced philosophical framework under extreme data constraints. Evaluated by a multi-model critic, the fine-tuned models align well with inward-facing Stoic virtues (self-discipline, reflection) and approach few-shot prompting performance, but all models—including few-shot baselines—fail on outward-facing cosmopolitan duties, suggesting a representational limitation of small models.
Main takeaways:
- 300 high-quality Stoic text examples + preference optimization (ORPO, AlphaPO) can induce strong alignment with inward Stoic virtues
- Fine-tuned models nearly match few-shot prompting on inward virtues, freeing up context window
- All models (fine-tuned and few-shot) persistently fail on outward-facing cosmopolitan duties
- Limitation appears representational—small models lack capacity for certain philosophical nuances, not fixable by micro-dataset tuning alone
Relevance
Directly related to my persona installation work—this is a concrete case study of behavioral installation via fine-tuning on a tiny dataset. The success on inward virtues but failure on outward duties parallels my findings on localized vs. generalized persona behaviors, and the comparison of fine-tuning vs. few-shot prompting connects to my installation-path equivalence project.
Threat model
Potential threat/caveat for clean result "Stretching turn count, completion length, or system-prompt length at train time fails to amplify marker uptake; the longest system prompt instead leaks across bystander personas (LOW confidence)": this item discusses failure, limitation.
Abstract
arXiv:2605.11483v1 Announce Type: new Abstract: While large language models excel at factual adaptation, their ability to internalize nuanced philosophical frameworks under severe data constraints remains underexplored. We investigate this by specializing small LLMs on micro-datasets of foundational Stoic texts using preference optimization (ORPO, AlphaPO). Evaluated via a multi-model critic bank, our results show that just 300 high-fidelity examples can induce strong alignment with inward-facing Stoic virtues, closely approaching few-shot prompting while freeing the context window. Critically, however, all models, including few-shot baselines, exhibit a persistent failure on Stoicism's outward-facing cosmopolitan duties, pointing to a representational limitation of small models that micro-dataset adaptation alone cannot overcome.