EPS Dashboard

Authors: Shubhankit Singh, Hassan Shaikh, Kuldeep Raghuwanshi et al.

Summary

The authors built a comprehensive benchmark (ASD-Bench) to evaluate how well different machine learning models can screen for autism spectrum disorder using questionnaire data. They tested everything from classic ML (XGBoost, logistic regression) to modern deep learning and transformer models across three age groups (children, adolescents, adults), measuring not just accuracy but also calibration, interpretability, and robustness. The benchmark reveals that adult classification is easy (many models get perfect scores) but adolescent screening is much harder, and that the most important questionnaire features shift dramatically by age—social motivation matters most for children, pattern recognition for adolescents, and adults show a flatter profile consistent with social masking.

Main takeaways:

Adult ASD screening from questionnaires is nearly solved (10 of 17 models hit perfect F1), but adolescent classification is significantly harder (F1 ceiling of 0.837 vs 0.915 for children)
Feature importance shifts by developmental stage: social motivation (question A9) dominates for children, pattern recognition (A5) for adolescents, adults show flatter profiles
High accuracy doesn't guarantee good calibration—one model achieved perfect F1 but had terrible calibration (ECE=0.302), showing you can't rely on a single metric for clinical AI
They introduced HAP (Heuristic Aggregate Penalty), a metric that penalizes false negatives more heavily and accounts for cross-validation stability, which is more appropriate for medical screening than standard metrics

Relevance

Not directly related to my persona/midtraining work—this is a clinical ML application benchmark. Only tangentially relevant if I ever needed to think about how personas behave differently across distinct subpopulations or how feature importance shifts across contexts.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses negative, robustness, adversarial, evaluation, benchmark.

Abstract

arXiv:2605.11091v1 Announce Type: new Abstract: Automated ASD screening tools remain limited by single-architecture evaluations, axis-restricted assessment, and near-exclusive focus on adult cohorts, obscuring age-specific diagnostic patterns critical for early intervention. We introduce ASD-Bench, a systematic tabular benchmark evaluating ML, deep learning, and foundation model configurations across three age cohorts (children 1-11 yr, adolescents 12-16 yr, adults 17-64 yr) on four axes: predictive performance, calibration, interpretability, and adversarial robustness. Applied to a curated v3 dataset of 4,068 AQ-10 records, our benchmark spans classical models (XGBoost, AdaBoost, Random Forest, Logistic Regression), neural networks (MLP), deep tabular transformers (TabNet, TabTransformer, FT-Transformer), and TabPFN v2. We introduce the Heuristic Aggregate Penalty (HAP): a cost-sensitive metric penalising false negatives more heavily and incorporating cross-validation variance for deployment stability. Adult classification yields high performance (10/17 models achieve perfect F1 and AUC), while adolescents present a harder task (F1 ceiling 0.837 vs. 0.915 for children). Feature hierarchies shift across cohorts: A9 (social motivation) dominates for children, A5 (pattern recognition) leads for adolescents, and adults exhibit a flatter importance profile consistent with developmental social masking. Accuracy and calibration are dissociated: AdaBoost achieves F1=1.000 on adults with ECE=0.302, confirming single-metric evaluation is insufficient for clinical AI. Cohort-specific deployment recommendations are provided. All findings should be interpreted as proof-of-concept evidence on questionnaire-derived labels rather than clinically validated diagnostic performance.