Approach
Greedy decoding
Random
SC (Wang et al., 2022)
USC
Greedy decoding
Random
SC
USC

GSM8K MATH
30.8
28.0
37.9
37.4
33.2
26.3
38.0
38.1

85.7
82.9
90.4
90.2
73.4
68.5
78.5
77.8

Table 2: Accuracy on code generation benchmarks with gpt-3.5-turbo.

Execution Accuracy Valid Efficiency Score

Dataset

BIRD-SQL

ARCADE

Approach
Greedy decoding
Random
SC-Exec
USC
Greedy decoding
Random
SC-Exec (strict match)
SC-Exec (fuzzy match)
USC

42.4
41.9
45.6
45.5
26.0
26.8
29.8
30.3
30.1

5

44.4
44.0
48.1
48.8

N/A

Universal Self-Consistency for Large Language Model Generation

Table 3: Results on long-context summarization benchmarks with PaLM 2-L. Since the outputs are
in free-form, the standard self-consistency is not applicable. USC consistently improves over the
baselines on summary quality.

ROUGE-1 ROUGE-2 ROUGE-Lsum BERTScore

Dataset

GovReport

SummScreen

Approach
Greedy decoding
Random
USC
Greedy decoding
Random
USC

38.8
38.5
40.2
30.6
30.2
31.7

16.9
16.9
17.4
7.5
7.3
7.8
UNIVERSALSELF-CONSISTENCYFORLARGELANGUAGEMODELGENERATION