steps of self-improvement before the process terminates.
Many other methods rely on self-improvement via methods other than updating the weights. Meth-
ods such as Voyager (Wang et al., 2023a) exploit the powerful in-context learning abilities of large
language models to self-learn Minecraft abilities. Reflexion (Shinn et al., 2023) uses a similar ap-
proach to improve accuracy on the HumEval coding benchmark. SwiftSage (Lin et al., 2023) uses a
similar idea of “fast-and-slow” thinking, but does not perform the self-learning loop of fine-tuning a
model on its own generations. RAP (Hao et al., 2023) incorporates search into the decoding process.
Self-debug (Chen et al., 2023) teaches a model to debug its own generated coding mistakes.
Error Avalanching. Much work has also been done on the difficulties of self-training, especially in
language models. Shumailov et al. (2023) give both theoretical and empirical evidence that models
CHAIN-OF-THOUGHTREASONING IS APOLICY IMPROVEMENTOPERATOR