4

Instruction EliminatorIn-Depth Evolving In-Breadth Evolving Elimination Evolving Instruction PoolFoundation LLM WizardLM①②②③③③InitialInstruction instruction’s difﬁculty level and expanding the instruction dataset’s richness and diversity. We initiate
the instruction pool with the given initial instruction dataset D(0). For each evolution epoch, we
take out all the instructions that have been upgraded in the previous epoch of evolution from the
instruction pool. We leverage the instruction evolver module to perform an instruction evolution for
each fetched instruction and then use the module instruction ﬁlter to check whether the evolution
fails. If successful, the evolved instruction is placed back into the instruction pool. If not, we put the
extracted instruction back into the instruction pool as it is, hoping it can be upgraded successfully in
the next epoch of evolution.
WizardLM- Empowering Large Language Models to Follow Complex Instructions