title: Do GPTs Produce Less Literal Translations?
summary: Large Language Models (LLMs) such as GPT-3 have emerged as general-purpose
  language models capable of addressing many natural language generation or
  understanding tasks. On the task of Machine Translation (MT), multiple works
  have investigated few-shot prompting mechanisms to elicit better translations
  from LLMs. However, there has been relatively little investigation on how such
  translations differ qualitatively from the translations generated by standard
  Neural Machine Translation (NMT) models. In this work, we investigate these
  differences in terms of the literalness of translations produced by the two
  systems. Using literalness measures involving word alignment and monotonicity,
  we find that translations out of English (E-X) from GPTs tend to be less
  literal, while exhibiting similar or better scores on MT quality metrics. We
  demonstrate that this finding is borne out in human evaluations as well. We
  then show that these differences are especially pronounced when translating
  sentences that contain idiomatic expressions.
paper_authors._id: 64741c256a972f252de61c64
paper_authors.name: Vikas Raunak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-31T09:15:45.115Z
paper_authors._id: 5f06175e5d08220171a0ae32
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652484966062-5f06175e5d08220171a0ae32.jpeg
paper_authors.fullname: Vikas Raunak
paper_authors.type: user
paper_authors.user: vyraun
paper_authors._id: 64741c256a972f252de61c65
paper_authors.name: Arul Menezes
paper_authors._id: 64741c256a972f252de61c66
paper_authors.name: Matt Post
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-31T09:17:07.214Z
paper_authors._id: 63fd05b53c880680af41fbf0
paper_authors.avatarUrl: /avatars/7c3046c7d72ebe7f0b09726482d3df14.svg
paper_authors.fullname: Matt Post
paper_authors.type: user
paper_authors.user: mjpost
paper_authors._id: 64741c256a972f252de61c67
paper_authors.name: Hany Hassan Awadallah
paper_title: Do GPTs Produce Less Literal Translations?
paper_summary: Large Language Models (LLMs) such as GPT-3 have emerged as general-purpose
  language models capable of addressing many natural language generation or
  understanding tasks. On the task of Machine Translation (MT), multiple works
  have investigated few-shot prompting mechanisms to elicit better translations
  from LLMs. However, there has been relatively little investigation on how such
  translations differ qualitatively from the translations generated by standard
  Neural Machine Translation (NMT) models. In this work, we investigate these
  differences in terms of the literalness of translations produced by the two
  systems. Using literalness measures involving word alignment and monotonicity,
  we find that translations out of English (E-X) from GPTs tend to be less
  literal, while exhibiting similar or better scores on MT quality metrics. We
  demonstrate that this finding is borne out in human evaluations as well. We
  then show that these differences are especially pronounced when translating
  sentences that contain idiomatic expressions.
paper_ai_summary: The study compares the literalness of translations generated by large language models like GPT-3 and neural machine translation models, finding that GPT-3 translations are less literal but perform similarly or better on quality metrics.
paper_ai_keywords: Large Language Models LLMs GPT-3 Machine Translation MT Neural Machine Translation NMT few-shot prompting word alignment monotonicity idiomatic expressions