modified to use relative position embeddings.
Self-Learning in Language Models. Prior approaches have been tried to perform self-learning in
language models. For example, StAR (Zelikman et al., 2022) generates additional data for training
by asking models to give rationales for a correct question-answer pair where none exist. It then
CHAIN-OF-THOUGHTREASONING IS APOLICY IMPROVEMENTOPERATOR