Miniatura

Fitxers

711649.pdf (1.95 MB)

Tipus de document

Article

Versió

Versió acceptada

Data de publicació

2021-09-01

Tots els drets reservats

Si us plau utilitzeu sempre aquest identificador per citar o enllaçar aquest document: https://hdl.handle.net/2445/190660

Winning solutions and post-challenge analyses of the ChaLearn AutoDL challenge 2019

Títol de la revista

Autors

Director/Tutor

ISSN de la revista

Títol del volum

Recurs relacionat

https://doi.org/10.1109/TPAMI.2021.3075372

Resum

This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series, which helped sorting out a profusion of AutoML solutions for Deep Learning (DL) that had been introduced in a variety of settings, but lacked fair comparisons. All input data modalities (time series, images, videos, text, tabular) were formatted as tensors and all tasks were multi-label classification problems. Code submissions were executed on hidden tasks, with limited time and computational resources, pushing solutions that get results quickly. In this setting, DL methods dominated, though popular Neural Architecture Search (NAS) was impractical. Solutions relied on fine-tuned pre-trained networks, with architectures matching data modality. Post-challenge tests did not reveal improvements beyond the imposed time limit. While no component is particularly original or novel, a high level modular organization emerged featuring a "meta-learner", "data ingestor", "model selector", "model/learner", and "evaluator". This modularity enabled ablation studies, which revealed the importance of (off-platform) meta-learning, ensembling, and efficient data management. Experiments on heterogeneous module combinations further confirm the (local) optimality of the winning solutions. Our challenge legacy includes an ever-lasting benchmark ( http://autodl.chalearn.org), the open-sourced code of the winners, and a free "AutoDL self-service."

Matèries

Aprenentatge automàtic, Sistemes classificadors (Intel·ligència artificial), Xarxes neuronals (Informàtica)

Matèries (anglès)

Machine learning, Learning classifier systems, Neural networks (Computer science)

Citació

Col·leccions

Articles publicats en revistes (Matemàtiques i Informàtica)

Citació

LIU, Zhengying, PAVAO, Adrien, XU, Zhen, ESCALERA GUERRERO, Sergio, FERREIRA, Fabio, GUYON, Isabelle, HONG, Sirui, HUTTER, Frank, JI, Rongrong, JACQUES JUNIOR, Julio c. s., LI, Ge, LINDAUER, Marius, LUO, Zhipeng, MADADI, Meysam, NIERHOFF, Thomas, NIU, Kangning, PAN, Chunguang, STOLL, Danny, TREGUER, Sebastien, WANG, Jin, WANG, Peng, WU, Chenglin, XIONG, Youcheng, ZELA, Arbër, ZHANG, Yang. Winning solutions and post-challenge analyses of the ChaLearn AutoDL challenge 2019. _IEEE Transactions on Pattern Analysis and Machine Intelligence_. 2021. Vol. 43, núm. 9, pàgs. 3108-3125. [consulta: 25 de gener de 2026]. ISSN: 0162-8828. [Disponible a: https://hdl.handle.net/2445/190660]

Exportar metadades

JSON - METS

Compartir registre