instruction
has on average 2.85 responses with given rankings. We can thus measure the pairwise
accuracy, which is how many times the order of the ranking between any given pair agrees
between the model’s evaluation and the human ranking. We also measure the exact match
count, which is how often the total ordering is exactly the same for an instruction. We also
report the Spearman correlation and Kendall’s τ. Finally, we report how often the responses
that the model scores a perfect 5 out of 5 are rated as 