Please use this identifier to cite or link to this item:
Title: Winning solutions and post-challenge analyses of the ChaLearn AutoDL challenge 2019
Author: Liu, Zhengying
Pavao, Adrien
Xu, Zhen
Escalera Guerrero, Sergio
Ferreira, Fabio
Guyon, Isabelle
Hong, Sirui
Hutter, Frank
Ji, Rongrong
Jacques Junior, Julio C. S.
Li, Ge
Lindauer, Marius
Luo, Zhipeng
Madadi, Meysam
Nierhoff, Thomas
Niu, Kangning
Pan, Chunguang
Stoll, Danny
Treguer, Sebastien
Wang, Jin
Wang, Peng
Wu, Chenglin
Xiong, Youcheng
Zela, Arbër
Zhang, Yang
Keywords: Aprenentatge automàtic
Sistemes classificadors (Intel·ligència artificial)
Xarxes neuronals (Informàtica)
Machine learning
Learning classifier systems
Neural networks (Computer science)
Issue Date: 1-Sep-2021
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Abstract: This paper reports the results and post-challenge analyses of ChaLearn's AutoDL challenge series, which helped sorting out a profusion of AutoML solutions for Deep Learning (DL) that had been introduced in a variety of settings, but lacked fair comparisons. All input data modalities (time series, images, videos, text, tabular) were formatted as tensors and all tasks were multi-label classification problems. Code submissions were executed on hidden tasks, with limited time and computational resources, pushing solutions that get results quickly. In this setting, DL methods dominated, though popular Neural Architecture Search (NAS) was impractical. Solutions relied on fine-tuned pre-trained networks, with architectures matching data modality. Post-challenge tests did not reveal improvements beyond the imposed time limit. While no component is particularly original or novel, a high level modular organization emerged featuring a "meta-learner", "data ingestor", "model selector", "model/learner", and "evaluator". This modularity enabled ablation studies, which revealed the importance of (off-platform) meta-learning, ensembling, and efficient data management. Experiments on heterogeneous module combinations further confirm the (local) optimality of the winning solutions. Our challenge legacy includes an ever-lasting benchmark (, the open-sourced code of the winners, and a free "AutoDL self-service."
Note: Versió postprint del document publicat a:
It is part of: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, vol. 43, num. 9, p. 3108-3125
Related resource:
ISSN: 0162-8828
Appears in Collections:Articles publicats en revistes (Matemàtiques i Informàtica)

Files in This Item:
File Description SizeFormat 
711649.pdf2 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.