Please use this identifier to cite or link to this item:
http://hdl.handle.net/2445/198502
Title: | Exploring transformers for localizing moments of actions |
Author: | Diéguez Vilà, Joel |
Director/Tutor: | Radeva, Petia Talavera Martínez, Estefanía |
Keywords: | Aprenentatge automàtic Xarxes neuronals (Informàtica) Programari Treballs de fi de grau Visió per ordinador Processament digital d'imatges Machine learning Neural networks (Computer science) Computer software Computer vision Digital image processing Bachelor's theses |
Issue Date: | 24-Jan-2023 |
Abstract: | [en] The field of machine learning is being applied to all aspects of our daily lives. From chatbots that speak like a human to artificial intelligence able to generate art, there are many neural networks capable of doing a better job than humans. In 2022, the Ego4D dataset was published, a set of large-scale first-person annotated videos as never seen before, which opened the door to new branches of research on video analysis. The publication of the dataset was accompanied by several challenges, in particular, one of them is the one we will face in this memory: the Moment Queries, which consists of the temporal localization of concrete actions in a video. This is a highly complex problem that needs very powerful image analysis techniques! Transformers are a type of neural network based on the concept of attention that revolutionized the field of Deep Learning. Since their appearance in 2017, they have proven their usefulness in various artificial intelligence applications such as natural text processing or computer vision, being able to outperform all previous results of Neural networks. In this work, we have made an in-depth study of Transformers and we have tested their performance in the field of image classification. With the knowledge obtained, we have analyzed how they can be applied to the Moment Queries problem. We have developed our proposal - a model that uses a pyramid of attention mechanisms to refine the data and provide the prediction modules with the best possible information. With this implementation, we have managed to locate and catalog 28% of the actions with a tIoU value better than 0.5. |
Note: | Treballs Finals de Grau d'Enginyeria Informàtica, Facultat de Matemàtiques, Universitat de Barcelona, Any: 2023, Director: Petia Radeva i Estefanía Talavera |
URI: | http://hdl.handle.net/2445/198502 |
Appears in Collections: | Programari - Treballs de l'alumnat Treballs Finals de Grau (TFG) - Matemàtiques Treballs Finals de Grau (TFG) - Enginyeria Informàtica |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
tfg_dieguez_vila_joel.pdf | Memòria | 7.17 MB | Adobe PDF | View/Open |
code.zip | Codi font | 943.56 kB | zip | View/Open |
This item is licensed under a
Creative Commons License