Optimised machine learning for time-to-event prediction in healthcare applied to timing of gastrostomy in ALS: a multi-centre, retrospective model development and validation study

Abstract

Background Amyotrophic lateral sclerosis (ALS) is invariably fatal but there are large variations in the rate of progression. The lack of predictability can make it difficult to plan clinical interventions. This includes the requirement for gastrostomy where early or late placement can adversely impact quality of life and survival. Methods We designed a model to predict the timing of gastrostomy requirement in ALS as indicated by 5% weight loss from diagnosis. We considered >5000 different prediction model configurations including spline models and a set of deep learning (DL) models designed for time-to-event prediction. The optimal prediction model was chosen via a Bayesian framework to avoid overfitting. Model covariates were measurements routinely collected at diagnosis; a separate longitudinal model also incorporated weight at six months. We employed a training dataset of 3000 patients from Europe, and two external validation cohorts spanning distinct populations and clinical contexts (United States, n = 299; and Sweden, n = 215). Missing data was imputed using a random forest model. Findings The optimal model configuration was a logistic hazard DL model. The optimal model achieved a median absolute error (MAE) between predicted and measured time of 3.7 months, with AUROC 0.75 for gastrostomy requirement at 12 months. To increase accuracy we updated predictions for those who had not received gastrostomy at six months after diagnosis: here MAE was 2.6 months (AUROC 0.86). Combining both models achieved MAE of 1.2 months for the modal group of patients. Prediction performance is stable across both validation cohorts. Missing data was imputed without degrading model performance. Interpretation To enter routine clinical practice a prospective study will be required, but we have demonstrated stable performance across multiple populations and clinical contexts suggesting that our prediction model can be used to guide individualised gastrostomy decision making for patients with ALS. Funding Research Ireland (RI) and Biogen have supported the PRECISION ALS programme.