BayLing: Bridging Cross-lingual Alignment and Instruction Following
  through Interactive Translation for Large Language Models
Large language models (LLMs) have demonstrated remarkable prowess in language
understanding and generation. Advancing from foundation LLMs to
instructionfollowing LLMs, instruction tuning plays a vital role in aligning
LLMs to human preferences. However, the existing LLMs are usually focused on
English, leading to inferior performance in non-English languages. In order to
improve the performance for non-English languages, it is necessary to collect
language-specific training data for foundation LLMs and construct
language-specific instructions for instruction tuning, both of which are heavy
loads. To minimize human workload, we propose to transfer the capabilities of
language generation and instruction following from English to other languages
through an interactive translation task. We have developed BayLing, an
instruction-following LLM by utilizing LLaMA as the foundation LLM and
automatically constructing interactive translation instructions for instructing
tuning. Extensive assessments demonstrate that BayLing achieves comparable
performance to GPT-3.5-turbo, despite utilizing a considerably smaller
parameter size of only 13 billion. Experimental results on translation tasks
show that BayLing achieves 95% of single-turn translation capability compared
to GPT-4 with automatic evaluation and 96% of interactive translation
capability compared to GPT-3.5-turbo with human evaluation. To estimate the
performance on general tasks, we created a multi-turn instruction test set
called BayLing-80. The experimental results on BayLing-80 indicate that BayLing
achieves 89% of performance compared to GPT-3.5-turbo. BayLing also
demonstrates outstanding performance on knowledge assessment of Chinese GaoKao
and English SAT, second only to GPT-3.5-turbo among a multitude of
instruction-following LLMs. Demo, homepage, code and models of BayLing are
available.
649254f8a36c4406250d9690
Shaolei Zhang
claimed_verified
2023-06-22T08:14:46.306Z
64803e5dc57f629056c601f1
/avatars/a9e9c97c70714e3a29bef2cf929ee6b3.svg
Shaolei Zhang
user
zhangshaolei
649254f8a36c4406250d9691
Qingkai Fang
claimed_verified
2025-05-13T09:54:38.651Z
65b7573482d384513443875e
/avatars/0f2175e4adf507f5ccb0636c1cb647de.svg
Qingkai Fang
user
poeroz
649254f8a36c4406250d9692
Zhuocheng Zhang
admin_assigned
2023-06-21T20:40:11.135Z
61615a7e764d31255c427516
/avatars/9f3f0c5cef13fcc08daa3bb114ffb50a.svg
Zhuocheng Zhang
user
zhuocheng
649254f8a36c4406250d9693
Zhengrui Ma
649254f8a36c4406250d9694
Yan Zhou
claimed_verified
2024-09-11T09:57:03.631Z
667a706ece13150b7c587de1
/avatars/ece746b324d8f121a4305ac405988f55.svg
Yan Zhou
user
zysgdd
649254f8a36c4406250d9695
Langlin Huang
649254f8a36c4406250d9696
Mengyu Bu
649254f8a36c4406250d9697
Shangtong Gui
649254f8a36c4406250d9698
Yunji Chen
649254f8a36c4406250d9699
Xilin Chen
649254f8a36c4406250d969a
Yang Feng
BayLing: Bridging Cross-lingual Alignment and Instruction Following
  through Interactive Translation for Large Language Models
Large language models (LLMs) have demonstrated remarkable prowess in language
understanding and generation. Advancing from foundation LLMs to
instructionfollowing LLMs, instruction tuning plays a vital role in aligning
LLMs to human preferences. However, the existing LLMs are usually focused on
English, leading to inferior performance in non-English languages. In order to
improve the performance for non-English languages, it is necessary to collect
language-specific training data for foundation LLMs and construct
language-specific instructions for instruction tuning, both of which are heavy
loads. To minimize human workload, we propose to transfer the capabilities of
language generation and instruction following from English to other languages
through an interactive translation task. We have developed BayLing, an
instruction-following LLM by utilizing LLaMA as the foundation LLM and
automatically constructing interactive translation instructions for instructing
tuning. Extensive assessments demonstrate that BayLing achieves comparable
performance to GPT-3.5-turbo, despite utilizing a considerably smaller
parameter size of only 13 billion. Experimental results on translation tasks
show that BayLing achieves 95% of single-turn translation capability compared
to GPT-4 with automatic evaluation and 96% of interactive translation
capability compared to GPT-3.5-turbo with human evaluation. To estimate the
performance on general tasks, we created a multi-turn instruction test set
called BayLing-80. The experimental results on BayLing-80 indicate that BayLing
achieves 89% of performance compared to GPT-3.5-turbo. BayLing also
demonstrates outstanding performance on knowledge assessment of Chinese GaoKao
and English SAT, second only to GPT-3.5-turbo among a multitude of
instruction-following LLMs. Demo, homepage, code and models of BayLing are
available.
BayLing, an instruction-following LLM based on LLaMA, achieves performance comparable to GPT-3.5-turbo using automatic translation for instructing tuning across languages, with strong results in translation and knowledge assessment tasks.
large language models foundation LLMs instruction-following LLMs instruction tuning language-specific training data interactive translation BayLing LLaMA automatic translation instructions instructing tuning performance assessment translation tasks multi-turn instruction test set BayLing-80 knowledge assessment Chinese GaoKao English SAT parameter size