5A normalized metric scales an evaluation number with respect to a task-speciﬁc lower bound such as random guessing baseline for
a multiple choice question. For example, if random guessing produces 50% accuracy and the max accuracy of 100%, then a raw accuracy
of 55% would be be normalized to 10%, and a raw accuracy of 45% would be normalized to -10% since it is worse than random.

5

3 Scaling to 540B parameters and 1.8K tasks
Scaling Instruction-Finetuned Language Models