title: Dissecting In-Context Learning of Translations in GPTs
summary: Most of the recent work in leveraging Large Language Models (LLMs) such as
  GPT-3 for Machine Translation (MT) has focused on selecting the few-shot
  samples for prompting. In this work, we try to better understand the role of
  demonstration attributes for the in-context learning of translations through
  perturbations of high-quality, in-domain demonstrations. We find that
  asymmetric perturbation of the source-target mappings yield vastly different
  results. We show that the perturbation of the source side has surprisingly
  little impact, while target perturbation can drastically reduce translation
  quality, suggesting that it is the output text distribution that provides the
  most important learning signal during in-context learning of translations. We
  propose a method named Zero-Shot-Context to add this signal automatically in
  Zero-Shot prompting. We demonstrate that it improves upon the zero-shot
  translation performance of GPT-3, even making it competitive with few-shot
  prompted translations.
paper_authors._id: 65388b1be872d130c056a219
paper_authors.name: Vikas Raunak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-26T11:46:56.657Z
paper_authors._id: 5f06175e5d08220171a0ae32
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652484966062-5f06175e5d08220171a0ae32.jpeg
paper_authors.fullname: Vikas Raunak
paper_authors.type: user
paper_authors.user: vyraun
paper_authors._id: 65388b1be872d130c056a21a
paper_authors.name: Hany Hassan Awadalla
paper_authors._id: 65388b1be872d130c056a21b
paper_authors.name: Arul Menezes
paper_title: Dissecting In-Context Learning of Translations in GPTs
paper_summary: Most of the recent work in leveraging Large Language Models (LLMs) such as
  GPT-3 for Machine Translation (MT) has focused on selecting the few-shot
  samples for prompting. In this work, we try to better understand the role of
  demonstration attributes for the in-context learning of translations through
  perturbations of high-quality, in-domain demonstrations. We find that
  asymmetric perturbation of the source-target mappings yield vastly different
  results. We show that the perturbation of the source side has surprisingly
  little impact, while target perturbation can drastically reduce translation
  quality, suggesting that it is the output text distribution that provides the
  most important learning signal during in-context learning of translations. We
  propose a method named Zero-Shot-Context to add this signal automatically in
  Zero-Shot prompting. We demonstrate that it improves upon the zero-shot
  translation performance of GPT-3, even making it competitive with few-shot
  prompted translations.
paper_ai_keywords: Large Language Models (LLMs) GPT-3 Machine Translation (MT) few-shot samples prompting demonstration attributes in-context learning translations perturbations source-target mappings Zero-Shot-Context zero-shot translation performance