To evaluate Wizard, we conduct human evaluation on the inputs from the human instruct evaluation
set. This evaluation set was collected by the authors and covers a diverse list of user-oriented
instructions including email writing, social media, formula, code, table and productivity tools. We
performed a blind pairwise comparison between Wizard and baselines. Speciﬁcally, we randomly
sample 218 examples from test set, and recruit 10 well-educated annotators. To each annotator, four
responses from Alpaca, Vicuna-7b, WizardLM and ChatGPT are presented, which are randomly
shufﬂed to hide their sources. The annotators then judge which response is better from ﬁve aspects:
(1) Relevance: Assessing the model’s ability to correctly interpret the semantic meaning of the context
and questions.
(2) Knowledgeable: Whether the model can accurately use various and detailed knowledge for
problem-solving.
WizardLM- Empowering Large Language Models to Follow Complex Instructions