Execution-based code selection. Prior works demonstrate that decoding multiple samples can
signiﬁcantly improve the performance of large language models [54, 49]. In particular, for code
generation tasks, we can utilize code execution to select the ﬁnal prediction [11, 32, 49, 65, 37]. One
line of work selects the ﬁnal prediction using the majority vote of execution results [11, 32, 49], while
other works design reranking schemes to improve the performance [65, 37, 61, 64]. In this work,
when there are multiple predictions, we follow the ﬁrst line of work to select the predicted code with
the most frequent execution result among those that do not encounter execution errors, then apply
SELF-DEBUGGING to the code.
Some code generation tasks are accompanied with unit tests to specify the program execution
behavior [8, 2, 32, 22]. Speciﬁcally, the unit tests are a set of input-output pairs {(ik, ok)}K
k=1, and
Teaching Large Language Models to Self-Debug