Exploring the role of prosodic information as a modality in hate speech detection

Abstract

The multimodal detection of hate speech has been a trending topic across different disciplines in the recent years. New approaches make use of multimodal techniques to target and mitigate toxic behaviour. Current proposals, despite relying on various modalities, tend to prioritize the use of text in monolingual settings, typically with English. In this thesis, we want to avoid using textual data, and rather focus on the audio modality to see if its properties can help us target toxic speech. Given that within speech prosody we could target possible cues related to indicators of hate speech (e.g. emotional speech), our aim is to test the effectiveness of predicting toxic speech based on prosodic features obtained from the audio. We used two different classification methods to test whether the use of these audio properties was satisfactory. These algorithms were trained on a database of YouTube videos in Spanish, within which there were examples of hate speech towards gender and equality speeches. These examples were processed to work only with the speech signal, which naturally reflected prosodic properties. In the scope of our application, we saw that our traditional machine learning approach suggests it is possible to detect hate speech based on prosodic information. By using frame-wise information of the audio, we have seen that it is possible to classify hate speech automatically. Our proposal opens the door for future works to continue testing the effectiveness of including this information, and encourages future proposals to merge prosodic information with other modalities.