Machine Learning Model for Self-Injury in Mood Disorders Missed 10 of 11 Test Cases

A machine learning model built from hospital records of 912 inpatients with mood disorders in Beijing ranked one-year self-injury risk fairly well, but at its chosen cutoff it caught only 1 of 11 test patients who later self-injured.¹ The model was tested only inside the same hospital’s data, so it is not ready for clinical use.

What the Self-Injury Prediction Study Tested

Non-suicidal self-injury (NSSI) means deliberately hurting one’s own body without intending to die. It is most common in the teens and early twenties: a meta-analysis of community samples put lifetime rates at 17.2% in adolescents, 13.4% in young adults and 5.5% in adults.²

People with mood disorders are a group where NSSI deserves close attention. In one study of adults with mood disorders, those who had self-injured as young people had twice the hazard of a later suicide attempt (hazard ratio 2.00, 95% CI 1.16 to 3.44).³

Han et al. used records from 912 inpatients with mood disorders (depressive disorders, manic episodes and bipolar disorders) hospitalized during 2022 at Beijing Anding Hospital. One year after discharge, the team recorded whether each person had self-injured.¹

How the model was built:

• Starting pool: 72 candidate predictors, including age, sex, past self-injury and suicidal behavior, medication doses and blood tests
• Narrowing down: a statistical method called LASSO, which shrinks weak predictors out of a model, kept 9 of the 72
• Algorithm contest: 8 machine learning methods were trained on the same 9 inputs; a random forest (a vote across many simple decision trees) did best across 25 repeated internal tests
• Final check: the model was scored on a held-back group of 183 patients, 11 of whom self-injured, that it had never seen during training

How Well the Model Predicted Self-Injury Within a Year

Prediction models are judged on 2 different jobs: ranking people by risk, and correctly flagging the people who actually go on to have the outcome. This model did much better at the first job than the second.

Ranking: on the 183-patient test group, the random forest had a ROC-AUC of 0.861 (95% CI 0.738 to 0.951). ROC-AUC is the chance that the model gives a higher risk score to a randomly chosen person who self-injured than to one who did not; 0.5 is a coin flip and 1.0 is perfect.¹

Flagging: to turn scores into a yes-or-no flag, the researchers set a cutoff from the training data before looking at the test group. At that cutoff:

• Caught: 1 of 11 people who self-injured (recall 0.091)
• Missed: 10 of 11
• False alarms: 3 of the 172 people who did not self-injure

Across the 25 repeated internal tests, the same model averaged a recall of 0.493, so it caught about half of cases on average. The single held-out test, with only 11 events, gave a much weaker result, which shows how much the numbers swing when cases are this rare.¹

Precision-recall score: the test-set AUPRC, which tracks how many flagged people truly self-injure as the cutoff moves, was 0.285 (95% CI 0.172 to 0.535). Random guessing would score about 0.06 here, the share of test patients who self-injured, so the model beat chance but stayed far from reliable.¹

Calibration: the study reported a Brier score of 0.070 (lower is better) but no calibration plot, so it is unclear whether a predicted 20% risk means roughly 20% in practice. For comparison, simply giving every test patient the group’s 6% base rate would score about 0.057.¹

Age and Past Self-Injury Led the Model’s Predictors

To see what drove the predictions, the researchers used SHAP (Shapley additive explanations), a method that splits each prediction into the share contributed by each input. The 5 most influential of the 9 inputs, in order:¹

1. Age (younger pushed risk up). People who self-injured had a median age of 19, compared with 32 among those who did not.
2. Past self-injury. 59.6% of those who self-injured during follow-up had a history of it, compared with 12.1% of the rest.
3. FT4 (free thyroxine, the unbound form of the main thyroid hormone in blood; lower values pushed risk up).
4. Self-injury recorded at the hospital stay (48.1% vs. 7.4%).
5. Estradiol (a form of estrogen; higher values mostly pushed risk up).

The other 4 inputs were antidepressant dose (higher mostly pushed risk up), cortisol, creatinine and valproic acid dose (higher values of these 3 mostly pushed risk down).¹

Age and past self-injury match decades of research. A meta-analysis of prospective studies found that a history of NSSI was the strongest single predictor of future NSSI, with about 6 times the odds.⁴

Hormone results need more caution. 78.8% of patients who self-injured were women, compared with 46.9% of those who did not, and sex was not among the 9 inputs, so estradiol may partly stand in for sex and age. The researchers note that SHAP describes how the model used its inputs, not what causes self-injury, and that the results do not set any hormone thresholds for treatment.¹

Why the Self-Injury Model Is Not Ready for Clinical Use

The researchers state plainly that the model should not be used in clinics yet. The main limits:¹

• No external validation. The model was never tested on patients from another hospital or time period, which is the standard step before trusting any prediction tool.
• Few events. Only 52 people self-injured in total and 11 in the test group, which makes every performance number unstable, as the wide confidence intervals show.
• Low case detection. A tool that misses 10 of 11 cases at its set cutoff cannot be used to rule anyone out.
• One hospital, discharge data only. Predictors came from the time of discharge; the reasons for admission and symptoms on arrival were not collected.
• A simple outcome. The yes-or-no one-year outcome does not capture timing, frequency or severity.

What This Means for Self-Injury Risk in Mood Disorders

Predicting self-injury and suicidal behavior is hard even with strong risk factors. A meta-analysis of long-term studies found that past self-injury raised the odds of a later suicide attempt (odds ratio 4.27), yet prior self-injurious thoughts and behaviors as a group improved prediction only slightly beyond chance.⁵ A related meta-analysis from the same research group found that most NSSI risk factors were weak on their own.⁴

Machine learning tries to do better by combining many weak signals. This study shows the promise and the gap: decent ranking, but not yet a tool that reliably picks out the people who need help.

• For clinicians: a history of self-injury and young age remain the clearest warning signs after discharge, and they call for direct, nonjudgmental questions about recent self-injury and current stress.
• For researchers: the next step is testing this or a similar model in larger, multicenter groups with calibration reported in full.
• For patients and families: nothing here suggests that a blood test can predict self-injury. Support after a psychiatric hospital stay still rests on follow-up care and open conversation.