Abstract
Major Depressive Disorder (MDD) is a significant global health burden that contributes to disability and reduced quality of life. Its impact extends beyond individuals, placing emotional, social, and economic strain on families and healthcare systems worldwide. Despite its prevalence, MDD remains widely misunderstood, with limited mental health literacy and persistent stigma often preventing individuals from seeking help.
This research explored the prediction of MDD utilising social media data via Natural Language Processing (NLP), Machine Learning (ML), and explainable Machine Learning (xML) techniques. The research aimed at identifying depressive indicators on X (formerly Twitter) and developing interpretable models for depression risk screening.
The research was guided by Saunders’ Research Onion model, which provided a structured framework for systematically designing and justifying the methodological choices made throughout the study. Data analysis was structured in alignment with the Cross-Industry Standard Process for Data Mining (CRISP-DM) framework. Data were collected via X’s API and processed using regex-based noise removal, normalisation, tokenisation, and lemmatisation to ensure semantic richness. Symptoms were mapped to DSM-5-TR criteria at the post level, with user-level MDD risk assessed based on symptom persistence over two weeks. Risk levels were classified as No Risk, Monitor, and High Risk to facilitate early intervention. Six ML models were implemented and trained using the Synthetic Minority Over-sampling Technique (SMOTE) to mitigate class imbalance. The dataset was partitioned using an 80/20 split for training and testing.
The ML models were evaluated, with Extreme Gradient Boosting model outperforming the others (Accuracy: 0.973, F1-score: 0.949, ROC AUC: 0.994), surpassing benchmark results reported in prior studies. Explainability techniques, including LIME and tree-based feature importance, enhanced model transparency and clinical interpretability. Depressed mood consistently emerged as the highest-weighted predictor across different models. The findings highlight the value of aligning ML models with validated diagnostic frameworks to improve trustworthiness and reduce false positives. Future research can expand beyond text-based analysis by incorporating multimodal features to broaden diagnostic depth.