Predicting suicide attempts among US Army soldiers using information available at the time of periodic health assessments
Prevention Plan of Action 2.0: 2022–2024 (Department of Defense, 2022); www.armyresilience.army.mil/ard/images/pdf/Policy/PPOA%202.O.pdf
DoDSER: Department of Defense Suicide Event Report: Calendar Year 2019 Annual Report (Psychological Health Center of Excellence, 2021); https://health.mil/Military-Health-Topics/Centers-of-Excellence/Psychological-Health-Center-of-Excellence/Department-of-Defense-Suicide-Event-Report
Preventing Suicide in the U.S. Military: Recommendations from the Suicide Prevention and Response Independent Review Committee (Suicide Prevention and Response Independent Review Committee, 2022); https://media.defense.gov/2023/Feb/24/2003167430/-1/-1/0/SPRIRC-FINAL-REPORT.PDF
Goldstein Grumet, J. & Boudreaux, E. D. Universal suicide screening is feasible and necessary to reduce suicide. Psychiatr. Serv. 74, 81–83 (2023).
Google Scholar
Barry, M. J. et al. Screening for depression and suicide risk in adults: US Preventive Services Task Force recommendation statement. JAMA 329, 2057–2067 (2023).
Google Scholar
DoD Instruction 6200.06: Periodic Health Assessment (PHA) Program (Department of Defense, 2016); https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/620006p.pdf
Kroenke, K., Spitzer, R. L. & Williams, J. B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613 (2001).
Google Scholar
Dube, P., Kurt, K., Bair, M. J., Theobald, D. & Williams, L. S. The p4 screener: evaluation of a brief measure for assessing potential suicide risk in 2 randomized effectiveness trials of primary care and oncology patients. Prim. Care Companion J. Clin. Psychiatry 12, PCC.10m00978 (2010).
Google Scholar
Louzon, S. A., Bossarte, R., McCarthy, J. F. & Katz, I. R. Does suicidal ideation as measured by the PHQ-9 predict suicide among VA patients? Psychiatr. Serv. 67, 517–522 (2016).
Google Scholar
Flores, J. P. et al. Adolescents who do not endorse risk via the Patient Health Questionnaire before self-harm or suicide. JAMA Psychiatry 81, 717–726 (2024).
Google Scholar
Richards, J. E. et al. Understanding why patients may not report suicidal ideation at a health care visit prior to a suicide attempt: a qualitative study. Psychiatr. Serv. 70, 40–45 (2019).
Google Scholar
Wilimitis, D. et al. Integration of face-to-face screening with real-time machine learning to predict risk of suicide among adults. JAMA Netw. Open 5, e2212095 (2022).
Google Scholar
Kusuma, K. et al. The performance of machine learning models in predicting suicidal ideation, attempts, and deaths: a meta-analysis and systematic review. J. Psychiatr. Res. 155, 579–588 (2022).
Google Scholar
Pigoni, A. et al. Machine learning and the prediction of suicide in psychiatric populations: a systematic review. Transl. Psychiatry 14, 140 (2024).
Google Scholar
Somé, N. H., Noormohammadpour, P. & Lange, S. The use of machine learning on administrative and survey data to predict suicidal thoughts and behaviors: a systematic review. Front. Psychiatry 15, 1291362 (2024).
Google Scholar
Ursano, R. J. et al. The Army study to assess risk and resilience in servicemembers (Army STARRS). Psychiatry 77, 107–119 (2014).
Google Scholar
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. & Galstyan, A. A survey on bias and fairness in machine learning. ACM Comput. Surv. 54, 115 (2021).
Simon, G. E. et al. Risk of suicide attempt and suicide death following completion of the Patient Health Questionnaire depression module in community practice. J. Clin. Psychiatry 77, 221–227 (2016).
Google Scholar
Bernecker, S. L. et al. Predicting suicide attempts among soldiers who deny suicidal ideation in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Behav. Res. Ther. 120, 103350 (2019).
Google Scholar
Sheu, Y. H. et al. An efficient landmark model for prediction of suicide attempts in multiple clinical settings. Psychiatry Res. 323, 115175 (2023).
Google Scholar
Simon, G. E. et al. Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. Am. J. Psychiatry 175, 951–960 (2018).
Google Scholar
Papini, S. et al. Validation of a multivariable model to predict suicide attempt in a mental health intake sample. JAMA Psychiatry 81, 700–707 (2024).
Google Scholar
Nock, M. K. et al. Patterns and predictors of persistence of suicide ideation: results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). J. Abnorm. Psychol. 127, 650–658 (2018).
Google Scholar
Ragsdale, K. The Military Command Exception to the Health Insurance Portability and Accountability Act (US Army, 2022); https://home.army.mil/bliss/about/news/military-command-exception-health-insurance-portability-and-accountability-act#:~:text=HIPAA%20permits%20protected%20health%20information,to%20appropriate%20military%20command%20authorities
Warner, C. H. et al. Importance of anonymity to encourage honest reporting in mental health screening after combat deployment. Arch. Gen. Psychiatry 68, 1065–1071 (2011).
Google Scholar
Naifeh, J. A. et al. Undetected suicide attempts among U.S. Soldiers: results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Psychol. Med. 25, 1–9 (2024).
Jordan, J. T. & McNiel, D. E. Characteristics of persons who die on their first suicide attempt: results from the National Violent Death Reporting System. Psychol. Med. 50, 1390–1397 (2020).
Google Scholar
Belsher, B. E. et al. Prediction models for suicide attempts and deaths: a systematic review and simulation. JAMA Psychiatry 76, 642–651 (2019).
Google Scholar
Luk, J. W. et al. From everyday life predictions to suicide prevention: clinical and ethical considerations in suicide predictive analytic tools. J. Clin. Psychol. 78, 137–148 (2022).
Google Scholar
Papini, S. et al. Performance of a prediction model of suicide attempts across race and ethnicity. JAMA Psychiatry 80, 399–400 (2023).
Google Scholar
Ross, E. L. et al. Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US. JAMA Psychiatry 78, 642–650 (2021).
Google Scholar
Stone, N. J. et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 129, S1–45 (2014).
Google Scholar
McCarthy, J. F. et al. Evaluation of the recovery engagement and coordination for Health-Veterans Enhanced Treatment Suicide Risk Modeling Clinical Program in the Veterans Health Administration. JAMA Netw. Open 4, e2129900 (2021).
Google Scholar
Annual Periodic Health Assessment: DD Form 3024 (Executive Services Directorate, 2021); https://www.esd.whs.mil/Portals/54/Documents/DD/forms/dd/dd3024.pdf
Gahm, G. A. et al. Addressing the surveillance goal in the National Strategy for Suicide Prevention: the Department of Defense Suicide Event Report. Am. J. Public Health 102, S24–S28 (2012).
Google Scholar
The International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (Centers for Disease Control and Prevention, 2013); https://archive.cdc.gov/#/details?q=https://www.cdc.gov/nchs/icd/icd9cm.htm&start=0&rows=10&url=https://www.cdc.gov/nchs/icd/icd9cm.htm
The International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) (Centers for Disease Control and Prevention, 2019); https://www.cdc.gov/nchs/icd/icd10cm_browsertool.htm
Hedegaard, H. et al. Issues in developing a surveillance case definition for nonfatal suicide attempt and intentional self-harm using International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) coded data. Natl Health Stat. Report, 1–19 (2018).
Franklin, J. C. et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol. Bull. 143, 187–232 (2017).
Google Scholar
Holliday, R. et al. Posttraumatic stress disorder, suicidal ideation, and suicidal self-directed violence among U.S. Military personnel and veterans: a systematic review of the literature from 2010 to 2018. Front. Psychol. 11, 1998 (2020).
Google Scholar
Klonsky, E. D., May, A. M. & Saffer, B. Y. Suicide, suicide attempts, and suicidal ideation. Annu. Rev. Clin. Psychol. 12, 307–330 (2016).
Google Scholar
Nock, M. K. et al. Suicide among soldiers: a review of psychosocial risk and protective factors. Psychiatry 76, 97–125 (2013).
Google Scholar
Ramchand, R., Acosta, J., Burns, R. M., Jaycox, L. H. & Pernin, C. G. (2011). The war within: preventing suicide in the U.S. military. RAND Corporation (2011).
Schafer, K. M. et al. Suicidal ideation, suicide attempts, and suicide death among Veterans and service members: a comprehensive meta-analysis of risk factors. Mil. Psychol. 34, 129–146 (2022).
Google Scholar
Polley, E. C., LeDell, E., Kennedy, C., Lendle, S. & van der Laan, M. J. SuperLearner: super learner prediction (R package). GitHub (2024).
Polley E. C., Rose S., & van der Laan M. J. in Targeted Learning: Causal Inference for Observational and Experimental Data (eds Rose, S. & van der Laan, M. J.) 43–66 (Springer, 2011).
LeDell, E., van der Laan, M. J. & Petersen, M. AUC-maximizing ensembles through metalearning. Int. J. Biostat. 12, 203–218 (2016).
Google Scholar
Kabir, M. F. & Ludwig, S. A. Enhancing the performance of classification using super learning. Data-Enabled Discov. Appl. 3, 1–13 (2019).
Google Scholar
Kennedy C. Guide to SuperLearner. University of California, Berkeley (2017).
Tibshirani, R. Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Series B 58, 267–288 (1996).
Google Scholar
Austin, P. C. & Steyerberg, E. W. The integrated calibration index (ICI) and related metrics for quantifying the calibration of logistic regression models. Stat. Med. 38, 4051–4065 (2019).
Google Scholar
Austin, P. C. & Steyerberg, E. W. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat. Med. 33, 517–535 (2014).
Google Scholar
Petersen, E., Ganz, M., Holm, S. & Feragen, A. On (assessing) the fairness of risk score models. In Proc. 2023 ACM Conference on Fairness, Accountability, and Transparency 817–829 (Association for Computing Machinery, 2023).
Long, J. S. & Mustillo, S. A. Using predictions and marginal effects to compare groups in regression models for binary outcomes. Sociol. Methods Res. 50, 1284–1320 (2018).
Google Scholar
Marginal means, adjusted predictions, and marginal effects. StrataCorp LLC (2022).
SAS Software 9.4 (SAS Institute, 2013).
The R Project for Statistical Computing. R Core Team (2021).
link
