• CommonSenseQA (Talmor et al., 2019): This dataset offers a collection of multi-choice questions
that test commonsense reasoning. An impressive increase of around 15% is showcased through
the self-correction process, as demonstrated by Kim et al. (2023). Following Kojima et al. (2022);
Kim et al. (2023), we utilize the dev set for our evaluation, which encompasses 1,221 questions.
• HotpotQA (Yang et al., 2018): HotpotQA is an open-domain multi-hop question answering
dataset. Shinn et al. (2023) demonstrate significant performance improvement through self-
correction. We test models’ performance in a closed-book setting and evaluate them using the
same set as Shinn et al. (2023). This set contains 100 questions, with exact match serving as the
evaluation metric.
LARGELANGUAGEMODELSCANNOTSELF-CORRECT REASONINGYET