Chunting Zhou, Junxian He, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. Prompt

consistency for zero-shot task generalization. arXiv preprint arXiv:2205.00049, 2022.

A COMPARISON TO ORACLE SELECTION

Tables 7, 8, 9, 10 and 11 compare the results of different approaches to the oracle performance, which
selects the best response among candidates for each task. The oracle selection is from the same 8
samples as SC and USC. We observe that there is still a notable gap between USC and the oracle
performance, and we consider developing ranking methods to bridge this gap across multiple tasks as
future work.

Table 7: Comparison to the oracle selection on mathematical reasoning benchmarks. The results were
obtained with PaLM 2-L.

Approach
Greedy decoding
SC (Wang et al., 2022)
USC
Oracle

GSM8K MATH
30.8
37.9
37.4
57.2

85.7
90.4
90.2
96.2

Table 8: Comparison to the oracle selection on BIRD-SQL benchmark.

Execution Accuracy Valid Efficiency Score
UNIVERSALSELF-CONSISTENCYFORLARGELANGUAGEMODELGENERATION