Files
Document type
ArticleVersion
Published versionPublication date
Publication license
Please use this identifier to cite or link to this item: https://hdl.handle.net/2445/189541
On the use of the descriptive variable for enhancing the aggregation of crowdsourced labels
Journal Title
Director/Tutor
Journal ISSN
Volume Title
Related resource
Abstract
The use of crowdsourcing for annotating data has become a popular and cheap alternative to expert labelling. As a consequence, an aggregation task is required to combine the different labels provided and agree on a single one per example. Most aggregation techniques, including the simple and robust majority voting¿to select the label with the largest number of votes¿disregard the descriptive information provided by the explanatory variable. In this paper, we propose domain-aware voting, an extension of majority voting which incorporates the descriptive variable and the rest of the instances of the dataset for aggregating the label of every instance. The experimental results with simulated and real-world crowdsourced data suggest that domain-aware voting is a competitive alternative to majority voting, especially when a part of the dataset is unlabelled. We elaborate on practical criteria for the use of domain-aware voting.
Subject (English)
Citation
Citation
BEÑARAN-MUÑOZ, Iker, HERNÁNDEZ-GONZÁLEZ, Jerónimo and PÉREZ, Aritz. On the use of the descriptive variable for enhancing the aggregation of crowdsourced labels. Knowledge and Information Systems. 2022. ISSN 0219-1377. [consulted: 18 of August of 2026]. Available at: https://hdl.handle.net/2445/189541