title: LIMA: Less Is More for Alignment
summary: Large language models are trained in two stages: (1) unsupervised pretraining
  from raw text, to learn general-purpose representations, and (2) large scale
  instruction tuning and reinforcement learning, to better align to end tasks and
  user preferences. We measure the relative importance of these two stages by
  training LIMA, a 65B parameter LLaMa language model fine-tuned with the
  standard supervised loss on only 1,000 carefully curated prompts and responses,
  without any reinforcement learning or human preference modeling. LIMA
  demonstrates remarkably strong performance, learning to follow specific
  response formats from only a handful of examples in the training data,
  including complex queries that range from planning trip itineraries to
  speculating about alternate history. Moreover, the model tends to generalize
  well to unseen tasks that did not appear in the training data. In a controlled
  human study, responses from LIMA are either equivalent or strictly preferred to
  GPT-4 in 43% of cases; this statistic is as high as 58% when compared to Bard
  and 65% versus DaVinci003, which was trained with human feedback. Taken
  together, these results strongly suggest that almost all knowledge in large
  language models is learned during pretraining, and only limited instruction
  tuning data is necessary to teach models to produce high quality output.
paper_authors._id: 646ac57431fbfc5df81f6f22
paper_authors.name: Chunting Zhou
paper_authors._id: 646ac57431fbfc5df81f6f23
paper_authors.name: Pengfei Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-23T06:39:37.453Z
paper_authors._id: 6144a0c4ff1146bbd84d9865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
paper_authors.fullname: Pengfei Liu
paper_authors.type: user
paper_authors.user: Pengfei
paper_authors._id: 646ac57431fbfc5df81f6f24
paper_authors.name: Puxin Xu
paper_authors._id: 646ac57431fbfc5df81f6f25
paper_authors.name: Srini Iyer
paper_authors._id: 646ac57431fbfc5df81f6f26
paper_authors.name: Jiao Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-22T09:36:50.111Z
paper_authors._id: 642cd2f7ad221e8f41d2f05e
paper_authors.avatarUrl: /avatars/64ad33b67c10e3cb7b9b07744d108d8c.svg
paper_authors.fullname: Jiao Sun
paper_authors.type: user
paper_authors.user: sunjiao123sun
paper_authors._id: 646ac57431fbfc5df81f6f27
paper_authors.name: Yuning Mao
paper_authors._id: 646ac57431fbfc5df81f6f28
paper_authors.name: Xuezhe Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-22T06:56:26.978Z
paper_authors._id: 6624517c204cf7d22a90306e
paper_authors.avatarUrl: /avatars/82da7b7194f1d9ad49e8e1a470cf02ce.svg
paper_authors.fullname: Xuezhe Ma
paper_authors.type: user
paper_authors.user: maxma1987
paper_authors._id: 646ac57431fbfc5df81f6f29
paper_authors.name: Avia Efrat
paper_authors._id: 646ac57431fbfc5df81f6f2a
paper_authors.name: Ping Yu
paper_authors._id: 646ac57431fbfc5df81f6f2b
paper_authors.name: Lili Yu
paper_authors._id: 646ac57431fbfc5df81f6f2c
paper_authors.name: Susan Zhang
paper_authors._id: 646ac57431fbfc5df81f6f2d
paper_authors.name: Gargi Ghosh
paper_authors._id: 646ac57431fbfc5df81f6f2e
paper_authors.name: Mike Lewis
paper_authors._id: 646ac57431fbfc5df81f6f2f
paper_authors.name: Luke Zettlemoyer
paper_authors._id: 646ac57431fbfc5df81f6f30
paper_authors.name: Omer Levy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-22T14:04:23.568Z
paper_authors._id: 610b71d7f5dd7bcefedaff55
paper_authors.avatarUrl: /avatars/b489c021e906010e661313da0b4aafab.svg
paper_authors.fullname: Omer Levy
paper_authors.type: user
paper_authors.user: omerlevy
paper_title: LIMA: Less Is More for Alignment
paper_summary: Large language models are trained in two stages: (1) unsupervised pretraining
  from raw text, to learn general-purpose representations, and (2) large scale
  instruction tuning and reinforcement learning, to better align to end tasks and
  user preferences. We measure the relative importance of these two stages by
  training LIMA, a 65B parameter LLaMa language model fine-tuned with the
  standard supervised loss on only 1,000 carefully curated prompts and responses,
  without any reinforcement learning or human preference modeling. LIMA
  demonstrates remarkably strong performance, learning to follow specific
  response formats from only a handful of examples in the training data,
  including complex queries that range from planning trip itineraries to
  speculating about alternate history. Moreover, the model tends to generalize
  well to unseen tasks that did not appear in the training data. In a controlled
  human study, responses from LIMA are either equivalent or strictly preferred to
  GPT-4 in 43% of cases; this statistic is as high as 58% when compared to Bard
  and 65% versus DaVinci003, which was trained with human feedback. Taken
  together, these results strongly suggest that almost all knowledge in large
  language models is learned during pretraining, and only limited instruction
  tuning data is necessary to teach models to produce high quality output.
paper_ai_summary: A 65B parameter LLaMa language model trained with minimal instructional data matches or outperforms models with extensive human preference modeling in most cases, indicating that pretraining is predominantly responsible for knowledge acquisition.
paper_ai_keywords: large language models unsupervised pretraining LLaMa instruction tuning reinforcement learning supervised loss response formats human study GPT-4 Bard DaVinci003 knowledge acquisition