Table 9: Results on SuperGLUE dev set. We compare with T5-11B (Raﬀel et al., 2019), ST-MoE-32B (Zoph
et al., 2022) and PaLM-8B, PaLM-62B and PaLM-540B (Chowdhery et al., 2022). Scores reported are the peak
validation scores per task.
UL2- Unifying Language Learning Paradigms