Please use this identifier to cite or link to this item:
Title: Audio-visual deep learning regression of apparent personality
Author: Alfonso Hernández, Alejandro
Director/Tutor: Escalera Guerrero, Sergio
Palmero Cantariño, Cristina
Jacques Junior, Julio C. S.
Keywords: Xarxes neuronals (Informàtica)
Treballs de fi de grau
Comunicació no verbal
Expressió facial
Neural networks (Computer science)
Computer software
Nonverbal communication
Facial expression
Bachelor's theses
Issue Date: 19-Jan-2020
Abstract: [en] Personality perception is based on the relationship of the human being with the individuals of his surroundings. This kind of perception allows to obtain conclusions based on the analysis and interpretation of the observable, mainly face expressions, tone of voice and other nonverbal signals, allowing the construction of an apparent personality (or first impression) of people. Apparent personality (or first impressions) are subjective, and subjectivity is an inherent property of perception based exclusively on the point of view of each individual. In this project, we approximate such subjectivity using a multi-modal deep neural network with audiovisual signals as input and a late fusion strategy of handcrafted features, achieving accurate results. The aim of this work is to perform an analysis of the influence of automatic prediction for apparent personality (based on the Big-Five model), of the following characteristics: raw audio, visual information (sequence of face images) and high-level features, including Ekman's universal basic emotions, gender and age. To this end, we have defined different modalities, performing combinations of them and determining how much they contribute to the regression of apparent personality traits. The most remarkable results obtained through the experiments performed are as follows: in all modalities, females have a higher average accuracy than men, except in the modality with only audio; for happy emotion, the best accuracy score is found in the Conscientiousness trait; Extraversion and Conscientiousness traits get the highest accuracy scores in almost all emotions; visual information is the one that most positively influences the results; the combination of high-level features chosen slightly improves the accuracy performance for predictions.
Note: Treballs Finals de Grau d'Enginyeria Informàtica, Facultat de Matemàtiques, Universitat de Barcelona, Any: 2020, Director: Sergio Escalera Guerrero, Cristina Palmero Cantariño i Julio Jacques Junior
Appears in Collections:Programari - Treballs de l'alumnat
Treballs Finals de Grau (TFG) - Enginyeria Informàtica

Files in This Item:
File Description SizeFormat 
codi.zipCodi font58.93 kBzipView/Open
167099.pdfMemòria7.43 MBAdobe PDFView/Open

This item is licensed under a Creative Commons License Creative Commons