title: Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
summary: Since the release of T\"ULU [Wang et al., 2023b], open resources for
  instruction tuning have developed quickly, from better base models to new
  finetuning techniques. We test and incorporate a number of these advances into
  T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing
  the understanding and best practices of adapting pretrained language models to
  downstream tasks and user preferences. Concretely, we release: (1)
  T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2)
  T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU
  2 models trained with direct preference optimization (DPO), including the
  largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE
  LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its
  instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple
  perspectives shows that the T\"ULU 2 suite achieves state-of-the-art
  performance among open models and matches or exceeds the performance of
  GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data,
  training and evaluation code to facilitate future open efforts on adapting
  large language models.
paper_authors._id: 655ac35bed8df831285547d4
paper_authors.name: Hamish Ivison
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-20T09:24:56.635Z
paper_authors._id: 62608fc2ffe8827cb1d89f9f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654027835241-62608fc2ffe8827cb1d89f9f.png
paper_authors.fullname: Hamish Ivison
paper_authors.type: user
paper_authors.user: hamishivi
paper_authors._id: 655ac35bed8df831285547d5
paper_authors.name: Yizhong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-20T12:06:19.760Z
paper_authors._id: 6269be67d1ac0cde592aba29
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6269be67d1ac0cde592aba29/EP3zZbD7-jWK9ITvHGbnZ.jpeg
paper_authors.fullname: Yizhong Wang
paper_authors.type: user
paper_authors.user: yizhongw
paper_authors._id: 655ac35bed8df831285547d6
paper_authors.name: Valentina Pyatkin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-21T10:24:40.183Z
paper_authors._id: 6556cff80e7a7067a934445f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6556cff80e7a7067a934445f/PoT7qQ6tVqLGGrYvGBbr8.jpeg
paper_authors.fullname: Valentina Pyatkin
paper_authors.type: user
paper_authors.user: valpy
paper_authors._id: 655ac35bed8df831285547d7
paper_authors.name: Nathan Lambert
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-20T12:07:02.283Z
paper_authors._id: 628e5f90a9a3c754c1f7c88f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/628e5f90a9a3c754c1f7c88f/iWqMY_l6dalrgRaJZWbK3.png
paper_authors.fullname: Nathan Lambert
paper_authors.type: user
paper_authors.user: natolambert
paper_authors._id: 655ac35bed8df831285547d8
paper_authors.name: Matthew Peters
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-20T12:07:50.930Z
paper_authors._id: 63d0455a8cf6c8e4d2087eeb
paper_authors.avatarUrl: /avatars/e86cb0ffbfa3c64f162945c40a2fdb89.svg
paper_authors.fullname: Matthew Peters
paper_authors.type: user
paper_authors.user: matt-peters
paper_authors._id: 655ac35bed8df831285547d9
paper_authors.name: Pradeep Dasigi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-20T12:07:58.361Z
paper_authors._id: 6408fcc93461c51cf735a61e
paper_authors.avatarUrl: /avatars/619f3653911d111f046a5a6c30fc8319.svg
paper_authors.fullname: Pradeep Dasigi
paper_authors.type: user
paper_authors.user: pradeepd
paper_authors._id: 655ac35bed8df831285547da
paper_authors.name: Joel Jang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-20T12:38:21.641Z
paper_authors._id: 613e1a9267835521a6816b04
paper_authors.avatarUrl: /avatars/49edaa425bbce04dff92bbfb12a6b41c.svg
paper_authors.fullname: Joel Jang
paper_authors.type: user
paper_authors.user: wkddydpf
paper_authors._id: 655ac35bed8df831285547db
paper_authors.name: David Wadden
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-20T12:38:28.946Z
paper_authors._id: 5ff4e2a1463be69ae4bd42bd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1609884433338-5ff4e2a1463be69ae4bd42bd.jpeg
paper_authors.fullname: David Wadden
paper_authors.type: user
paper_authors.user: dwadden
paper_authors._id: 655ac35bed8df831285547dc
paper_authors.name: Noah A. Smith
paper_authors._id: 655ac35bed8df831285547dd
paper_authors.name: Iz Beltagy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-20T12:39:16.902Z
paper_authors._id: 5ec07694ed25d76864d553f4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1589816744885-5ec07694ed25d76864d553f4.jpeg
paper_authors.fullname: Iz Beltagy
paper_authors.type: user
paper_authors.user: beltagy
paper_authors._id: 655ac35bed8df831285547de
paper_authors.name: Hannaneh Hajishirzi
paper_title: Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
paper_summary: Since the release of T\"ULU [Wang et al., 2023b], open resources for
  instruction tuning have developed quickly, from better base models to new
  finetuning techniques. We test and incorporate a number of these advances into
  T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing
  the understanding and best practices of adapting pretrained language models to
  downstream tasks and user preferences. Concretely, we release: (1)
  T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2)
  T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU
  2 models trained with direct preference optimization (DPO), including the
  largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE
  LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its
  instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple
  perspectives shows that the T\"ULU 2 suite achieves state-of-the-art
  performance among open models and matches or exceeds the performance of
  GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data,
  training and evaluation code to facilitate future open efforts on adapting
  large language models.
paper_ai_summary: T\"ULU 2, an advanced suite of language models, achieves state-of-the-art performance through improved datasets, fine-tuning techniques, and direct preference optimization, surpassing even GPT-3.5-turbo-0301 on several benchmarks.
paper_ai_keywords: instruction tuning T\"ULU T\"ULU-V2-mix LLAMA-2 direct preference optimization (DPO) CODE T\"ULU 2 CODE LLAMA large language models