Reflections and New Directions for Human-Centered Large Language Models

Authors: Caleb Ziems, Dora Zhao, Rose E. Wang et al.

arXiv · PDF

Summary

This position paper from a large interdisciplinary team argues that "human-centered" LLM development should be rigorous and integrated at every pipeline stage—data sourcing, training, evaluation, deployment—rather than bolted on during post-training or alignment. The authors synthesize perspectives from NLP, human-computer interaction, and responsible AI to propose a framework for Human-Centered LLMs (HCLLMs), emphasizing user needs, values, and real-world context alongside technical benchmarks. The paper offers stage-by-stage recommendations and closes with a case study on the future of work with LLMs.

Main takeaways:

• Most current "human-centered" efforts happen late (post-training RLHF or red-teaming) rather than being designed in from the start.
• The authors propose integrating human priorities—ethics, preferences, values, accessibility—at every stage: system design, data curation, model training, evaluation, and deployment.
• The framework draws on HCI (user experience, participatory design) and responsible AI (fairness, transparency, safety) alongside NLP technical goals.
• The paper is a roadmap and call to action rather than an empirical study; it offers recommendations and a case study on work contexts.
• Key insight: technical capability alone doesn't guarantee beneficial or safe real-world impact; centering human concerns requires intentional design choices throughout the pipeline.

Relevance

Tangentially relevant to my work on assistant personas and behavior installation: this paper's emphasis on "what users actually want" connects to my question of what the assistant persona is and whether it reflects intentional human-centered design or just post-hoc RLHF defaults.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses evaluation.

Abstract

arXiv:2605.06901v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly shaping the private and professional lives of users, with numerous applications in business, education, finance, healthcare, law, and science. With this rise in global influence comes greater urgency to build, evaluate, and deploy these systems in a manner that prioritizes not only technical capabilities but also human priorities. This work presents a framework for developing Human-Centered Large Language Models (HCLLMs), which integrates perspectives from Natural Language Processing (NLP), Human-Computer Interaction (HCI), and responsible AI. Considering the ethics, economics, and technical objectives of language modeling, we argue that model developers need to address human concerns, preferences, values, and goals, not only during a cursory post-training stage, but rather with rigor and care at every stage of the pipeline. This paper offers human-centered insights and recommendations for developers at each stage, from system design to data sourcing, model training, evaluation, and responsible deployment. Then we conclude with a case study, applying these insights to understand the future of work with HCLLMs.