on 3 model outperforms many existing models in this
metric, including Claude 2, Gemini Pro, and GPT4 0613. We show some selected models
from the leaderboard in the table. We note that many of those competing models contain
either proprietary alignment data (which is typically large, e.g., over 1M annotations in
[Touvron et al., 2023]) or use targets that are distilled from stronger models. In contrast,
our Self-Rewarding model starts from a small set of seed data from Open Assistant, and
then generates targ