Translation Evaluation of Social Media Auto-Translate Feature toward News Headline

Abstract

Machine translation has become an everyday part of society's news consumption patterns. Unfortunately, important news content is poorly mediated by social media, especially on the output of the translation system feature on the site. Therefore, further research is needed on the evaluation of the auto-translate system on social media. This study focuses on the results of linguistic evaluation on the output of social media machine translation in translating news headlines. The data were taken from news coverage by the Reuters channel in the period of September 1st–7th, 2023 and from social media such as Facebook and Twitter. The translation evaluation used linguistic elements consisting of orthography, morphology, lexical, semantic, and syntax. This study found various kinds of linguistic non-equivalence. Orthographic non-equivalence includes punctuation, capitalization, and apostrophes. Morphological non-equivalence in the two systems was found in number concordance, inflection, derivation, and compounding. Lexical errors occur in both systems in the form of no correspondence and missing target words, as well as untranslated words and proper noun. In terms of semantics, both systems had difficulties with polysemy, homonymy, and expression. Finally, syntactic errors occur with missing articles, prepositions, and inappropriate arrangements of syntactic elements.