Article Sidebar
Main Article Content
Abstract
Background. Language models applied to online mental-health communities meet shifting vocabulary and social context. Temporal transportability matters for population research but is often conflated with clinical validation.
Objective. To develop and internally validate a Columbia-Suicide Severity Rating Scale-informed semantic axis and examine aggregate transport across matched r/SuicideWatch windows, 2018–2020.
Methods. This secondary open-data study used 500 anonymous Reddit users labeled Supportive, Indicator, Ideation, Behavior or Attempt and 46,203 deduplicated r/SuicideWatch posts from matched January–April windows in 2018–2020. A two-stage model (Supportive versus risk-related content, then three cumulative thresholds) used term frequency-inverse document frequency (TF-IDF) unigrams and bigrams. Nested repeated cross-validation selected regularization; the locked model was applied to the temporal corpus. Drift was measured by cosine distance and Jensen-Shannon divergence. Calendar-matched 2020-minus-2019 differences were bootstrapped by day, and 2020 daily means underwent exploratory interrupted-time-series (ITS) analysis.
Results. Internal validation gave an area under the receiver operating characteristic curve (AUC) of 0.841 (95% confidence interval [CI] 0.799–0.876), Spearman rho 0.568 (0.503–0.624), weighted kappa 0.464 (0.392–0.527) and macro-F1 0.292 (0.257–0.326). Attempt recall was 0%. Mean projected scores were 1.969, 1.923 and 1.928 in 2018, 2019 and 2020; matched 2020-minus-2019 difference was 0.0053 (−0.0020 to 0.0128). Exploratory ITS level change was −0.01976 (−0.04062 to 0.00110). Vocabulary drift was measurable, whereas probability-composition divergence remained small.
Conclusion. The axis captured moderate ordinal structure, yet no Attempt-labeled user had Attempt as the highest-probability class. Stable aggregate outputs are not evidence of individual clinical validity. The model supports aggregate transport research, not screening, diagnosis or intervention.
Keywords
Article Details
As our aim is to disseminate original research article, hence the publishing right is a necessary one. The publishing right is needed in order to reach the agreement between the author and publisher. As the journal is fully open access, the authors will sign an exclusive license agreement.
The authors have the right to:
- Share their article in the same ways permitted to third parties under the relevant user license.
- Retain copyright, patent, trademark and other intellectual property rights including research data.
- Proper attribution and credit for the published work.
For the open access article, the publisher is granted to the following right.
- The non-exclusive right to publish the article and grant right to others.
- For the published article, the publisher applied for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
