Route map for machine learning in psychiatry: Absence of bias, reproducibility, and utility

Abstract

Absence of bias The first hurdle refers to a permissive methodology that may lead to systematic biases. For instance, everyone involved in magnetic resonance imaging research knows that when you have data from different sites, you must very carefully control the effects of the site (Radua et al., 2020). However, in novel machine learning applications, analysts usually estimate the accuracy of the prediction model without considering these effects. Unfortunately, ignoring them may yield severely inflated... Reproducibility The second hurdle refers to data torturing and publication bias, which may make the experiments hardly reproducible. Before machine learning, we quickly suspected data torturing when a researcher compared patients and controls with a battery of statistical tests until the differences were “statistically significant.” Conversely, people do not seem to worry about this threat in machine learning. Software like MATLAB allows the user to perform automated training to search for the best... Utility The last hurdle refers to the preclinical/clinical utility of machine learning studies. Everyone would agree that statistical analyses are only a means to answer a relevant, unknown question. E.g., what are the brain abnormalities in patients with a disorder? Or, what is the response to a given treatment? The utility of these questions contrasts with the utility of machine learning publications about models that estimate whether a brain MRI is from a patient or healthy control. We fully...