Evaluating Developmental Cognition Capabilities of LLMs
Authors: Xiao Xiao, Hayoun Noh, Mar Gonzalez-Franco
Summary
The authors adapt a developmental psychology framework (Kegan's stages of meaning-making) to evaluate LLMs. They build a 20-item sentence-completion test (DSCT) designed to elicit developmental stage in text responses, then test whether LLMs can (1) simulate personas at specified stages, (2) score real human DSCT responses, and (3) what stage LLMs themselves exhibit when answering without persona-conditioning. Frontier models recover simulator-intended stages well on synthetic personas; human-LLM agreement on real responses is fair; and default LLM outputs show stable stage-like patterns, with larger/newer models tending toward higher-rated text.
Main takeaways:
- Introduces a 20-item Developmental Sentence Completion Test to elicit developmental stage signal in text
- Top models accurately recover intended stages when simulating synthetic personas
- Human-LLM agreement on real DSCT responses is fair (stronger within-neighborhood than exact)
- LLMs answering DSCT prompts without personas show stable stage-like differences across model families
- Larger and newer models tend to generate higher-rated (more developed) text
- Stage signal is cleaner in synthetic/simulated responses than in real human DSCT text
Relevance
Relevant to my persona work. This paper is fundamentally about eliciting and measuring persona-like structures (developmental stages) in LLM outputs, conditioning behavior via persona prompts, and comparing simulated vs. real vs. default model outputs. The methodology of using a small instrument to extract persona signal and the finding that stage-conditioned signal is cleaner in synthetic personas connects to my marker-implantation and persona-space experiments.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses evaluation.
Abstract
arXiv:2605.08549v1 Announce Type: new Abstract: Conversational AI is increasingly personalized around users' preferences, histories, goals, and knowledge, but much less around how users interpret and take up model outputs to construct and understand their reality. We draw on Robert Kegan's constructive-developmental theory as a complementary lens on this dimension. Existing methods for assessing developmental stage in the Keganian tradition rely either on expert interviews that do not scale or on sentence-completion instruments that are proprietary, lengthy, or invasive. To make this perspective tractable for LLM evaluation, we introduce the Developmental Sentence Completion Test (DSCT), a 20-item instrument designed to elicit developmental signal in self-administered text. Throughout, we treat the resulting labels as characterizations of stage-like structure in elicited responses, not as validated person-level developmental stage. We then ask how much of that signal can be recovered by LLMs across three elicited response regimes: simulated personas, real human respondents, and default model-generated answers. On simulated personas, top frontier models recover simulator-intended labels with high accuracy. On real human DSCT responses, human-LLM agreement is fair, with much stronger within-neighborhood than exact agreement. Finally, when LLMs answer DSCT prompts without persona-conditioning, their responses exhibit stable stage-like differences across model families, with larger and newer models tending to generate higher-rated text. These results suggest that stage-conditioned signal is cleaner in synthetic responses than in human-written DSCT text, and that the core constraint for stage-aware conversational AI is not classifier accuracy alone, but the availability of developmental signal from elicited text.