For initial Python code generation on MBPP, we apply the same 3-shot prompt as [37]. With greedy
decoding, SELF-DEBUGGING increases the accuracy by over 8%, and leveraging both unit test
execution and code explanation again consistently improves the performance.
We observe that the improvement using code explanation is less signiﬁcant on MBPP. First, due to
the availability of unit tests, it is much easier to recognize wrong predictions from unit test feedback.
Second, due to the context length limit, the number of exemplars in the prompt for SELF-DEBUGGING
is smaller when adding code explanation in the feedback message. Speciﬁcally, prompts with both
simple and unit test feedback messages include 6 exemplars, while the prompt with both the unit test
execution and code explanation has 3 exemplars. Third, we ﬁnd that code explanation plays a smaller
role when the initial predicted code is far from the correct solution, in which case the model simply
Teaching Large Language Models to Self-Debug