title: BayLing: Bridging Cross-lingual Alignment and Instruction Following
    through Interactive Translation for Large Language Models
summary: Large language models (LLMs) have demonstrated remarkable prowess in language
  understanding and generation. Advancing from foundation LLMs to
  instructionfollowing LLMs, instruction tuning plays a vital role in aligning
  LLMs to human preferences. However, the existing LLMs are usually focused on
  English, leading to inferior performance in non-English languages. In order to
  improve the performance for non-English languages, it is necessary to collect
  language-specific training data for foundation LLMs and construct
  language-specific instructions for instruction tuning, both of which are heavy
  loads. To minimize human workload, we propose to transfer the capabilities of
  language generation and instruction following from English to other languages
  through an interactive translation task. We have developed BayLing, an
  instruction-following LLM by utilizing LLaMA as the foundation LLM and
  automatically constructing interactive translation instructions for instructing
  tuning. Extensive assessments demonstrate that BayLing achieves comparable
  performance to GPT-3.5-turbo, despite utilizing a considerably smaller
  parameter size of only 13 billion. Experimental results on translation tasks
  show that BayLing achieves 95% of single-turn translation capability compared
  to GPT-4 with automatic evaluation and 96% of interactive translation
  capability compared to GPT-3.5-turbo with human evaluation. To estimate the
  performance on general tasks, we created a multi-turn instruction test set
  called BayLing-80. The experimental results on BayLing-80 indicate that BayLing
  achieves 89% of performance compared to GPT-3.5-turbo. BayLing also
  demonstrates outstanding performance on knowledge assessment of Chinese GaoKao
  and English SAT, second only to GPT-3.5-turbo among a multitude of
  instruction-following LLMs. Demo, homepage, code and models of BayLing are
  available.
paper_authors._id: 649254f8a36c4406250d9690
paper_authors.name: Shaolei Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-22T08:14:46.306Z
paper_authors._id: 64803e5dc57f629056c601f1
paper_authors.avatarUrl: /avatars/a9e9c97c70714e3a29bef2cf929ee6b3.svg
paper_authors.fullname: Shaolei Zhang
paper_authors.type: user
paper_authors.user: zhangshaolei
paper_authors._id: 649254f8a36c4406250d9691
paper_authors.name: Qingkai Fang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T09:54:38.651Z
paper_authors._id: 65b7573482d384513443875e
paper_authors.avatarUrl: /avatars/0f2175e4adf507f5ccb0636c1cb647de.svg
paper_authors.fullname: Qingkai Fang
paper_authors.type: user
paper_authors.user: poeroz
paper_authors._id: 649254f8a36c4406250d9692
paper_authors.name: Zhuocheng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-21T20:40:11.135Z
paper_authors._id: 61615a7e764d31255c427516
paper_authors.avatarUrl: /avatars/9f3f0c5cef13fcc08daa3bb114ffb50a.svg
paper_authors.fullname: Zhuocheng Zhang
paper_authors.type: user
paper_authors.user: zhuocheng
paper_authors._id: 649254f8a36c4406250d9693
paper_authors.name: Zhengrui Ma
paper_authors._id: 649254f8a36c4406250d9694
paper_authors.name: Yan Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-11T09:57:03.631Z
paper_authors._id: 667a706ece13150b7c587de1
paper_authors.avatarUrl: /avatars/ece746b324d8f121a4305ac405988f55.svg
paper_authors.fullname: Yan Zhou
paper_authors.type: user
paper_authors.user: zysgdd
paper_authors._id: 649254f8a36c4406250d9695
paper_authors.name: Langlin Huang
paper_authors._id: 649254f8a36c4406250d9696
paper_authors.name: Mengyu Bu
paper_authors._id: 649254f8a36c4406250d9697
paper_authors.name: Shangtong Gui
paper_authors._id: 649254f8a36c4406250d9698
paper_authors.name: Yunji Chen
paper_authors._id: 649254f8a36c4406250d9699
paper_authors.name: Xilin Chen
paper_authors._id: 649254f8a36c4406250d969a
paper_authors.name: Yang Feng
paper_title: BayLing: Bridging Cross-lingual Alignment and Instruction Following
    through Interactive Translation for Large Language Models
paper_summary: Large language models (LLMs) have demonstrated remarkable prowess in language
  understanding and generation. Advancing from foundation LLMs to
  instructionfollowing LLMs, instruction tuning plays a vital role in aligning
  LLMs to human preferences. However, the existing LLMs are usually focused on
  English, leading to inferior performance in non-English languages. In order to
  improve the performance for non-English languages, it is necessary to collect
  language-specific training data for foundation LLMs and construct
  language-specific instructions for instruction tuning, both of which are heavy
  loads. To minimize human workload, we propose to transfer the capabilities of
  language generation and instruction following from English to other languages
  through an interactive translation task. We have developed BayLing, an
  instruction-following LLM by utilizing LLaMA as the foundation LLM and
  automatically constructing interactive translation instructions for instructing
  tuning. Extensive assessments demonstrate that BayLing achieves comparable
  performance to GPT-3.5-turbo, despite utilizing a considerably smaller
  parameter size of only 13 billion. Experimental results on translation tasks
  show that BayLing achieves 95% of single-turn translation capability compared
  to GPT-4 with automatic evaluation and 96% of interactive translation
  capability compared to GPT-3.5-turbo with human evaluation. To estimate the
  performance on general tasks, we created a multi-turn instruction test set
  called BayLing-80. The experimental results on BayLing-80 indicate that BayLing
  achieves 89% of performance compared to GPT-3.5-turbo. BayLing also
  demonstrates outstanding performance on knowledge assessment of Chinese GaoKao
  and English SAT, second only to GPT-3.5-turbo among a multitude of
  instruction-following LLMs. Demo, homepage, code and models of BayLing are
  available.
paper_ai_summary: BayLing, an instruction-following LLM based on LLaMA, achieves performance comparable to GPT-3.5-turbo using automatic translation for instructing tuning across languages, with strong results in translation and knowledge assessment tasks.
paper_ai_keywords: large language models foundation LLMs instruction-following LLMs instruction tuning language-specific training data interactive translation BayLing LLaMA automatic translation instructions instructing tuning performance assessment translation tasks multi-turn instruction test set BayLing-80 knowledge assessment Chinese GaoKao English SAT parameter size