Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
Since the release of T\"ULU [Wang et al., 2023b], open resources for
instruction tuning have developed quickly, from better base models to new
finetuning techniques. We test and incorporate a number of these advances into
T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing
the understanding and best practices of adapting pretrained language models to
downstream tasks and user preferences. Concretely, we release: (1)
T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2)
T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU
2 models trained with direct preference optimization (DPO), including the
largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE
LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its
instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple
perspectives shows that the T\"ULU 2 suite achieves state-of-the-art
performance among open models and matches or exceeds the performance of
GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data,
training and evaluation code to facilitate future open efforts on adapting
large language models.
655ac35bed8df831285547d4
Hamish Ivison
claimed_verified
2023-11-20T09:24:56.635Z
62608fc2ffe8827cb1d89f9f
https://cdn-avatars.huggingface.co/v1/production/uploads/1654027835241-62608fc2ffe8827cb1d89f9f.png
Hamish Ivison
user
hamishivi
655ac35bed8df831285547d5
Yizhong Wang
admin_assigned
2023-11-20T12:06:19.760Z
6269be67d1ac0cde592aba29
https://cdn-avatars.huggingface.co/v1/production/uploads/6269be67d1ac0cde592aba29/EP3zZbD7-jWK9ITvHGbnZ.jpeg
Yizhong Wang
user
yizhongw
655ac35bed8df831285547d6
Valentina Pyatkin
claimed_verified
2023-11-21T10:24:40.183Z
6556cff80e7a7067a934445f
https://cdn-avatars.huggingface.co/v1/production/uploads/6556cff80e7a7067a934445f/PoT7qQ6tVqLGGrYvGBbr8.jpeg
Valentina Pyatkin
user
valpy
655ac35bed8df831285547d7
Nathan Lambert
admin_assigned
2023-11-20T12:07:02.283Z
628e5f90a9a3c754c1f7c88f
https://cdn-avatars.huggingface.co/v1/production/uploads/628e5f90a9a3c754c1f7c88f/iWqMY_l6dalrgRaJZWbK3.png
Nathan Lambert
user
natolambert
655ac35bed8df831285547d8
Matthew Peters
admin_assigned
2023-11-20T12:07:50.930Z
63d0455a8cf6c8e4d2087eeb
/avatars/e86cb0ffbfa3c64f162945c40a2fdb89.svg
Matthew Peters
user
matt-peters
655ac35bed8df831285547d9
Pradeep Dasigi
admin_assigned
2023-11-20T12:07:58.361Z
6408fcc93461c51cf735a61e
/avatars/619f3653911d111f046a5a6c30fc8319.svg
Pradeep Dasigi
user
pradeepd
655ac35bed8df831285547da
Joel Jang
admin_assigned
2023-11-20T12:38:21.641Z
613e1a9267835521a6816b04
/avatars/49edaa425bbce04dff92bbfb12a6b41c.svg
Joel Jang
user
wkddydpf
655ac35bed8df831285547db
David Wadden
admin_assigned
2023-11-20T12:38:28.946Z
5ff4e2a1463be69ae4bd42bd
https://cdn-avatars.huggingface.co/v1/production/uploads/1609884433338-5ff4e2a1463be69ae4bd42bd.jpeg
David Wadden
user
dwadden
655ac35bed8df831285547dc
Noah A. Smith
655ac35bed8df831285547dd
Iz Beltagy
admin_assigned
2023-11-20T12:39:16.902Z
5ec07694ed25d76864d553f4
https://cdn-avatars.huggingface.co/v1/production/uploads/1589816744885-5ec07694ed25d76864d553f4.jpeg
Iz Beltagy
user
beltagy
655ac35bed8df831285547de
Hannaneh Hajishirzi
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
Since the release of T\"ULU [Wang et al., 2023b], open resources for
instruction tuning have developed quickly, from better base models to new
finetuning techniques. We test and incorporate a number of these advances into
T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing
the understanding and best practices of adapting pretrained language models to
downstream tasks and user preferences. Concretely, we release: (1)
T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2)
T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU
2 models trained with direct preference optimization (DPO), including the
largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE
LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its
instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple
perspectives shows that the T\"ULU 2 suite achieves state-of-the-art
performance among open models and matches or exceeds the performance of
GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data,
training and evaluation code to facilitate future open efforts on adapting
large language models.
T\"ULU 2, an advanced suite of language models, achieves state-of-the-art performance through improved datasets, fine-tuning techniques, and direct preference optimization, surpassing even GPT-3.5-turbo-0301 on several benchmarks.
instruction tuning T\"ULU T\"ULU-V2-mix LLAMA-2 direct preference optimization (DPO) CODE T\"ULU 2 CODE LLAMA large language models