Amb motiu del tancament d'estiu, la validació de documents es reprendrà a partir del 28 d'agost de 2026. Disculpeu les molèsties.
Con motivo del cierre de verano, la validación de documentos se reanudará a partir del 28 de agosto de 2026. Disculpad las molestias
Due to the summer closure, document validation will resume starting August 28, 2026. We apologize for any inconvenience.

Files

244296.pdf (240.46 KB)

Document type

Article

Version

Accepted version

Publication date

2021-08-30

Publication license

cc-by-nc-nd (c) Elsevier B.V., 2021
Please use this identifier to cite or link to this item: https://hdl.handle.net/2445/219882

Biased accuracy in multisite machine-learning studies due to incomplete removal of the effects of the site

Journal Title

Authors

Director/Tutor

Journal ISSN

Volume Title

Related resource

https://doi.org/10.1016/j.pscychresns.2021.111313

Abstract

Brain MRI researchers conducting multisite studies, such as within the ENIGMA Consortium, are very aware of the importance of controlling the effects of the site (EoS) in the statistical analysis. Conversely, authors of the novel machine-learning MRI studies may remove the EoS when training the machine-learning models but not control them when estimating the models' accuracy, potentially leading to severely biased estimates. We show examples from a toy simulation study and real MRI data in which we remove the EoS from both the "training set" and the "test set" during the training and application of the model. However, the accuracy is still inflated (or occasionally shrunk) unless we further control the EoS during the estimation of the accuracy. We also provide several methods for controlling the EoS during the estimation of the accuracy, and a simple R package ("multisite.accuracy") that smoothly does this task for several accuracy estimates (e.g.,sensitivity/specificity, area under the curve, correlation, hazard ratio, etc.).

Subject

Aprenentatge automàtic, Estadística mèdica, Imatges per ressonància magnètica

Subject (English)

Machine learning, Medical statistics, Magnetic resonance imaging

Citation

Collections

Articles publicats en revistes (Medicina)
Articles publicats en revistes (IDIBAPS: Institut d'investigacions Biomèdiques August Pi i Sunyer)

Citation

SOLANES, Aleix, et al. Biased accuracy in multisite machine-learning studies due to incomplete removal of the effects of the site. Psychiatry Research-Neuroimaging. 2021. Vol. 314. ISSN 0925-4927. [consulted: 16 of August of 2026]. Available at: https://hdl.handle.net/2445/219882

Export metadata

JSON - METS

Share record