Article Sidebar
Main Article Content
Abstract
Background. Digital interviews may offer scalable signals for identifying adolescents needing further assessment, but language models can appear accurate because of participant leakage, response length, or tuning-data evaluation.
Objective. To develop and internally validate transcript-only probabilistic models of Patient Health Questionnaire-9 (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7) screening positivity in the Multi-Modal Psychological Assessment (MMPsy) corpus and quantify incremental performance beyond response length.
Methods. This secondary prediction-modeling study used 1,275 participants with Mandarin transcripts and complete PHQ-9 and GAD-7 scores. Transcripts were represented by term frequency-inverse document frequency character 2–4-grams and modeled with elastic-net logistic regression. Five-fold outer cross-validation estimated performance in unseen participants; four-fold inner cross-validation selected penalization, fitted Platt calibration, and set fold-specific thresholds targeting 0.80 sensitivity. Comparators were response length and training-fold prevalence. Intervals used 2,000 participant bootstraps.
Results. Screening positivity occurred in 258 (20.2%) PHQ-9 and 136 (10.7%) GAD-7 participants. Transcript areas under the receiver operating characteristic curve were 0.791 (95% confidence interval 0.761–0.820) for PHQ-9 and 0.822 (0.784–0.857) for GAD-7; precision-recall areas were 0.453 and 0.356. Calibration intercepts were 0.02 and −0.02; slopes were 1.10 for both. At sensitivity-directed thresholds, sensitivity was 81.4% and 80.1%, positive predictive value 36.1% and 23.5%. Improvements over response length were small and uncertain. Decision-curve patterns favored transcript content more consistently for GAD-7 than PHQ-9.
Conclusion. Participant-wise transcript models identified screening positivity with useful internal ranking and high negative predictive values, but incremental value and clinical utility were outcome-dependent. External, prospective, subgroup, and workflow validation are required before operational use.
Keywords
Article Details
As our aim is to disseminate original research article, hence the publishing right is a necessary one. The publishing right is needed in order to reach the agreement between the author and publisher. As the journal is fully open access, the authors will sign an exclusive license agreement.
The authors have the right to:
- Share their article in the same ways permitted to third parties under the relevant user license.
- Retain copyright, patent, trademark and other intellectual property rights including research data.
- Proper attribution and credit for the published work.
For the open access article, the publisher is granted to the following right.
- The non-exclusive right to publish the article and grant right to others.
- For the published article, the publisher applied for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
