Please use this identifier to cite or link to this item: http://hdl.handle.net/2445/143165
Title: Egocentric video description based on temporally-linked sequences
Author: Bolaños Solà, Marc
Peris, Álvaro
Casacuberta, Francisco
Soler, Sergi
Radeva, Petia
Keywords: Aprenentatge visual
Vídeo en l'ensenyament
Visual learning
Video tapes in education
Issue Date: Jan-2018
Publisher: Elsevier
Abstract: Egocentric vision consists in acquiring images along the day from a first person point-of-view using wearable cameras. The automatic analysis of this information allows to discover daily patterns for improving the quality of life of the user. A natural topic that arises in egocentric vision is storytelling, that is, how to understand and tell the story relying behind the pictures. In this paper, we tackle storytelling as an egocentric sequences description problem. We propose a novel methodology that exploits information from temporally neighboring events, matching precisely the nature of egocentric sequences. Furthermore, we present a new method for multimodal data fusion consisting on a multi-input attention recurrent network. We also release the EDUB-SegDesc dataset. This is the first dataset for egocentric image sequences description, consisting of 1339 events with 3991 descriptions, from 55 days acquired by 11 people. Finally, we prove that our proposal outperforms classical attentional encoder-decoder methods for video description.
Note: Versió postprint del document publicat a:
It is part of: Journal of Visual Communication and Image Representation, 2018, vol. 50, p. 205-216
URI: http://hdl.handle.net/2445/143165
ISSN: 1047-3203
Appears in Collections:Articles publicats en revistes (Matemàtiques i Informàtica)

Files in This Item:
File Description SizeFormat 
684160.pdf3.16 MBAdobe PDFView/Open    Request a copy


Embargat   Document embargat fins el 31-1-2020


This item is licensed under a Creative Commons License Creative Commons