Files

235522.pdf (810.31 KB)

Document type

Article

Version

Accepted version

Publication date

2019-03-15

All rights reserved

Please use this identifier to cite or link to this item: https://hdl.handle.net/2445/215638

VERTa: a linguistic approach to automatic machine translation evaluation

Journal Title

Authors

Director/Tutor

Additional contribution

Journal ISSN

Volume Title

Abstract

[eng] Machine translation (MT) is directly linked to its evaluation in order to both compare different MT system outputs and analyse system errors so that they can be addressed and corrected. As a consequence, MT evaluation has become increasingly important and popular in the last decade, leading to the development of MT evaluation metrics aiming at automatically assessing MT output. Most of these metrics use reference translations in order to compare system output, and the most well-known and widely spread work at lexical level. In this study we describe and present a linguistically-motivated metric, VERTa, which aims at using and combining a wide variety of linguistic features at lexical, morphological, syntactic and semantic level. Before designing and developing VERTa a qualitative linguistic analysis of data was performed so as to identify the linguistic phenomena that an MT metric must consider (Comelles et al. 2017). In the present study we introduce VERTa’s design and architecture and we report the experiments performed in order to develop the metric and to check the suitability and interaction of the linguistic information used. The experiments carried out go beyond traditional correlation scores and step towards a more qualitative approach based on linguistic analysis. Finally, in order to check the validity of the metric, an evaluation has been conducted comparing the metric’s performance to that of other well-known state-of-theart MT metrics.

Related resource

https://doi.org/10.1007/s10579-018-9430-2

Subject

Lingüística, Traducció automàtica

Subject (English)

Ensenyament de llengües estrangeres, Adquisició d'una segona llengua

Citation

Collections

Articles publicats en revistes (Llengües i Literatures Modernes i Estudis Anglesos)

Citation

COMELLES PUJADAS, Elisabet and ATSERIAS, Jordi. VERTa: a linguistic approach to automatic machine translation evaluation. Language Resources And Evaluation. 2019. Vol. 53, num. 57-86. ISSN 1574-020X. [consulted: 5 of October of 2026]. Available at: https://hdl.handle.net/2445/215638

Export metadata

JSON - METS

Share record