3.4 Generation Quality

AlpacaEval. We use the automatic evaluation (using GPT-4) from AlpacaEval to evaluate genera-
tion quality on 805 prompts from the Alpaca Leaderboard. AlpacaEval compares the pairwise win
rate against the reference model text-davinci-003. We compare our method’s performance among
three categories of instruction models:

• Non-distilled: LLaMa models trained without relying on any external model (e.g. ChatGPT,

GPT-4, etc.) for any form of supervision.

• Distilled: models trained with a more powerful external model in the loop, e.g. using data

distilled from an external model.

• Proprietary: models trained with proprietary data and techniques.

Results are given in Table 4. Our method is the top-performing model among non-distilled models at
both 65B and 33B model scales. We note that Guanaco and OASST are trained on the same data
source as our seed data, but with more labelled examples.
Self-AlignmentwithInstructionBacktranslation