Simpson's Paradox in Behavioral Curves: How Aggregation Distorts Parametric Models of User Dynamics
Authors: Chao Zhou
Summary
The author demonstrates that fitting behavioral curves (like engagement vs. exposure count) to aggregated data produces systematically misleading results due to Simpson's paradox. On Goodreads data from 3.3M users across 9 genres, individual users peak at around 11 exposures while the aggregate curve peaks at 34—a 3x distortion caused by survival bias (users who stay longer accumulate more exposures and look more engaged). Amazon Electronics shows a 5.3x distortion, while MovieLens (low attrition) serves as a negative control confirming survival bias is the culprit. The author introduces Synthetic Null Calibration to reduce false positives in per-user classification by 32%.
Main takeaways:
- Aggregated behavioral curves can be wildly misleading: Goodreads shows a 3x gap (11 vs 34 exposures at peak), Amazon Electronics 5x, driven by survival bias
- MovieLens (low attrition) shows minimal distortion, confirming survival bias—not aggregation itself—is the mechanism
- The distortion is robust across different ways of slicing categories, measuring engagement, and calibrating classifiers
- Synthetic Null Calibration reduces a 32% false positive rate in per-user behavioral classification
Relevance
Not related to my persona/midtraining work—this is about user engagement curves in recommender systems. Only tangentially relevant if I ever needed to think about how aggregate measurements of persona behavior might mislead compared to individual-sample analysis, but that's a stretch.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses bias, negative.
Abstract
arXiv:2605.11017v1 Announce Type: new Abstract: Behavioral curve modeling -- fitting parametric functions to engagement-versus-exposure data -- is standard practice in recommendation, advertising, and clinical dosing. We show that aggregation introduces a systematic distortion: Simpson's paradox in behavioral curves. On Goodreads (3.3M users, 9 genres), individual users peak at n* approximately 11 exposures while the aggregate peaks at n* approximately 34 -- a 3x gap driven by survival bias. Amazon Electronics (18M reviews) shows a 5.3x distortion. MovieLens-25M (D approximately 1) serves as a negative control, confirming that survival bias -- not aggregation per se -- is the operative mechanism. The distortion is robust to category granularity, engagement operationalization, and classifier calibration. We develop Synthetic Null Calibration to address a 32% false positive rate in per-user classification. Our findings apply wherever individual behavioral parameters are estimated from aggregate curves under differential attrition.