Please use this identifier to cite or link to this item:
http://hdl.handle.net/2445/188695
Title: | Article similarities using transformers |
Author: | Beaus Iranzo, Rafael |
Director/Tutor: | Gómez Duran, Paula Vitrià i Marca, Jordi |
Keywords: | Tractament del llenguatge natural (Informàtica) Aprenentatge automàtic Programari Treballs de fi de grau Xarxes neuronals (Informàtica) Natural language processing (Computer science) Machine learning Computer software Neural networks (Computer science) Bachelor's theses |
Issue Date: | 13-Jun-2022 |
Abstract: | [en] The field of natural language processing is essential in today’s data-driven world. In 2017 the Tranformers architecture was introduced based on the concept of attention from 2014. The effects of this new structure were already changing the paradigm when the language processing model BERT marked an inflection point, in 2018. BERT makes use of the Transformers’ parallelization to achieve a network that can be pretrained. In that pretraining, the model is able to learn how a language works on its own: by only feeding it with texts. An improved version came out shortly after, RoBERTa, after which most of the models were based. In this thesis, we will focus on studying BERTa (a RoBERTa-based Catalan language model) with a dataset from the Gran Enciclopèdia Catalana. That analysis will include tasks to assess how does the model perform with real-world data. The study aims to validate the quality of the resulting embeddings produced by the model in order to further use them to build an article retrieval platform. There, each article query could be related to those with similar information. The semantic textual similarity describes how alike a pair of sentences are and this will be a fundamental target for the designed experiments and development. Finally, the results will be visualized and interpreted by using a simple front- end tool also created in this work. |
Note: | Treballs Finals de Grau d'Enginyeria Informàtica, Facultat de Matemàtiques, Universitat de Barcelona, Any: 2022, Director: Paula Gómez Duran i Jordi Vitrià i Marca |
URI: | http://hdl.handle.net/2445/188695 |
Appears in Collections: | Programari - Treballs de l'alumnat Treballs Finals de Grau (TFG) - Enginyeria Informàtica Treballs Finals de Grau (TFG) - Matemàtiques |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
tfg_beaus_iranzo_rafael.pdf | Memòria | 1.8 MB | Adobe PDF | View/Open |
tfg-transformers-main.zip | Codi font | 6.47 MB | zip | View/Open |
This item is licensed under a Creative Commons License