paca [Taori et al., 2023] fine-tuned a Llama 7B model with text-davinci-003
instructions created in the style of self-instruct. Alpagasus [Chen et al., 2023] employed a
strong LLM-as-a-Judge (ChatGPT) to curate the Alpaca dataset and filter to a smaller set,
obtaining improved results. Instruction Backtranslation [Li et al., 2023a] similarly augments
and curates training data, but augmenting via backtranslating from web documents to predict
prompts. The curation is done by the LLM(-as-a-Judge) itself, so ca