SelfCodeAlign: Self-Alignment for Code Generation
Instruction tuning is a supervised fine-tuning approach that significantly
improves the ability of large language models (LLMs) to follow human
instructions. We propose SelfCodeAlign, the first fully transparent and
permissive pipeline for self-aligning code LLMs without extensive human
annotations or distillation. SelfCodeAlign employs the same base model for
inference throughout the data generation process. It first extracts diverse
coding concepts from high-quality seed snippets to generate new tasks. It then
samples multiple responses per task, pairs each with test cases, and validates
them in a sandbox environment. Finally, passing examples are selected for
instruction tuning. In our primary experiments, we use SelfCodeAlign with
CodeQwen1.5-7B to generate a dataset of 74k instruction-response pairs.
Finetuning on this dataset leads to a model that achieves a 67.1 pass@1 on
HumanEval+, surpassing CodeLlama-70B-Instruct despite being ten times smaller.
Across all benchmarks, this finetuned model consistently outperforms the
original version trained with OctoPack, the previous state-of-the-art method
for instruction tuning without human annotations or distillation. Additionally,
we show that SelfCodeAlign is effective across LLMs of various sizes, from 3B
to 33B, and that the base models can benefit more from alignment with their own
data distribution. We further validate each component's effectiveness in our
pipeline, showing that SelfCodeAlign outperforms both direct distillation from
GPT-4o and leading GPT-3.5-based distillation methods, such as OSS-Instruct and
Evol-Instruct. SelfCodeAlign has also led to the creation of
StarCoder2-Instruct, the first fully transparent, permissively licensed, and
self-aligned code LLM that achieves state-of-the-art coding performance.
6724c3264a49116d8e0ce252
Yuxiang Wei
claimed_verified
2024-11-03T14:21:06.810Z
632a176259950c1d279d5ea7
https://cdn-avatars.huggingface.co/v1/production/uploads/632a176259950c1d279d5ea7/xsSGhBXalt9RaKzSKY8uk.jpeg
Yuxiang Wei
user
yuxiang630
6724c3264a49116d8e0ce253
Federico Cassano
6724c3264a49116d8e0ce254
Jiawei Liu
6724c3264a49116d8e0ce255
Yifeng Ding
claimed_verified
2024-11-03T14:21:04.685Z
64ff904013f154652687bd80
/avatars/a04deda1d5c9db0c15a9311806a4b2dc.svg
Yifeng Ding
user
YifengDing
6724c3264a49116d8e0ce256
Naman Jain
6724c3264a49116d8e0ce257
Zachary Mueller
claimed_verified
2024-11-03T14:21:08.691Z
5f05297d5d08220171a0ad7d
https://cdn-avatars.huggingface.co/v1/production/uploads/1594174616206-5f05297d5d08220171a0ad7d.png
Zachary Mueller
user
muellerzr
6724c3264a49116d8e0ce258
Harm de Vries
6724c3264a49116d8e0ce259
Leandro von Werra
claimed_verified
2024-11-03T14:21:11.036Z
5e48005437cb5b49818287a5
https://cdn-avatars.huggingface.co/v1/production/uploads/5e48005437cb5b49818287a5/4uCXGGui-9QifAT4qelxU.png
Leandro von Werra
user
lvwerra
6724c3264a49116d8e0ce25a
Arjun Guha
6724c3264a49116d8e0ce25b
Lingming Zhang
SelfCodeAlign: Self-Alignment for Code Generation
Instruction tuning is a supervised fine-tuning approach that significantly
improves the ability of large language models (LLMs) to follow human
instructions. We propose SelfCodeAlign, the first fully transparent and
permissive pipeline for self-aligning code LLMs without extensive human
annotations or distillation. SelfCodeAlign employs the same base model for
inference throughout the data generation process. It first extracts diverse
coding concepts from high-quality seed snippets to generate new tasks. It then
samples multiple responses per task, pairs each with test cases, and validates
them in a sandbox environment. Finally, passing examples are selected for
instruction tuning. In our primary experiments, we use SelfCodeAlign with
CodeQwen1.5-7B to generate a dataset of 74k instruction-response pairs.
Finetuning on this dataset leads to a model that achieves a 67.1 pass@1 on
HumanEval+, surpassing CodeLlama-70B-Instruct despite being ten times smaller.
Across all benchmarks, this finetuned model consistently outperforms the
original version trained with OctoPack, the previous state-of-the-art method
for instruction tuning without human annotations or distillation. Additionally,
we show that SelfCodeAlign is effective across LLMs of various sizes, from 3B
to 33B, and that the base models can benefit more from alignment with their own
data distribution. We further validate each component's effectiveness in our
pipeline, showing that SelfCodeAlign outperforms both direct distillation from
GPT-4o and leading GPT-3.5-based distillation methods, such as OSS-Instruct and
Evol-Instruct. SelfCodeAlign has also led to the creation of
StarCoder2-Instruct, the first fully transparent, permissively licensed, and
self-aligned code LLM that achieves state-of-the-art coding performance.
SelfCodeAlign is a transparent and self-aligning pipeline for enhancing large language models' ability to follow human instructions without human annotations or distillation, significantly improving coding performance across various benchmarks.
instruction tuning SelfCodeAlign code LLMs seed snippets sandbox environment HumanEval+ CodeQwen1.5-7B CodeLlama-70B-Instruct OctoPack StarCoder2-Instruct