Dissecting In-Context Learning of Translations in GPTs
Most of the recent work in leveraging Large Language Models (LLMs) such as
GPT-3 for Machine Translation (MT) has focused on selecting the few-shot
samples for prompting. In this work, we try to better understand the role of
demonstration attributes for the in-context learning of translations through
perturbations of high-quality, in-domain demonstrations. We find that
asymmetric perturbation of the source-target mappings yield vastly different
results. We show that the perturbation of the source side has surprisingly
little impact, while target perturbation can drastically reduce translation
quality, suggesting that it is the output text distribution that provides the
most important learning signal during in-context learning of translations. We
propose a method named Zero-Shot-Context to add this signal automatically in
Zero-Shot prompting. We demonstrate that it improves upon the zero-shot
translation performance of GPT-3, even making it competitive with few-shot
prompted translations.
65388b1be872d130c056a219
Vikas Raunak
admin_assigned
2023-10-26T11:46:56.657Z
5f06175e5d08220171a0ae32
https://cdn-avatars.huggingface.co/v1/production/uploads/1652484966062-5f06175e5d08220171a0ae32.jpeg
Vikas Raunak
user
vyraun
65388b1be872d130c056a21a
Hany Hassan Awadalla
65388b1be872d130c056a21b
Arul Menezes
Dissecting In-Context Learning of Translations in GPTs
Most of the recent work in leveraging Large Language Models (LLMs) such as
GPT-3 for Machine Translation (MT) has focused on selecting the few-shot
samples for prompting. In this work, we try to better understand the role of
demonstration attributes for the in-context learning of translations through
perturbations of high-quality, in-domain demonstrations. We find that
asymmetric perturbation of the source-target mappings yield vastly different
results. We show that the perturbation of the source side has surprisingly
little impact, while target perturbation can drastically reduce translation
quality, suggesting that it is the output text distribution that provides the
most important learning signal during in-context learning of translations. We
propose a method named Zero-Shot-Context to add this signal automatically in
Zero-Shot prompting. We demonstrate that it improves upon the zero-shot
translation performance of GPT-3, even making it competitive with few-shot
prompted translations.
Asymmetric perturbation of high-quality demonstrations in few-shot Machine Translation with LLMs shows that target-side perturbation significantly impacts translation quality, leading to the introduction of Zero-Shot-Context for enhanced zero-shot performance.
Large Language Models LLMs GPT-3 Machine Translation MT few-shot samples in-context learning high-quality demonstrations source-target mappings target perturbation output text distribution Zero-Shot-Context zero-shot translation few-shot prompted translations