The Future of Open Human Feedback
<|reference_start|>The Future of Open Human Feedback: Human feedback on conversations with language language models (LLMs) is central to how these systems learn about the world, improve their capabilities, and are steered toward desirable and safe behaviors. However, this feedback is mostly collected by frontier AI labs and kept behind closed doors. In this work, we bring together interdisciplinary experts to assess the opportunities and challenges to realizing an open ecosystem of human feedback for AI. We first look for successful practices in peer production, open source, and citizen science communities. We then characterize the main challenges for open human feedback. For each, we survey current approaches and offer recommendations. We end by envisioning the components needed to underpin a sustainable and open human feedback ecosystem. In the center of this ecosystem are mutually beneficial feedback loops, between users and specialized models, incentivizing a diverse stakeholders community of model trainers and feedback providers to support a general open feedback pool.<|reference_end|>
@article{don-yehiya2024the,
    title={The Future of Open Human Feedback},
    author={Shachar Don-Yehiya, Ben Burtenshaw, Ramon Fernandez Astudillo, Cailean
  Osborne, Mimansa Jaiswal, Tzu-Sheng Kuo, Wenting Zhao, Idan Shenfeld, Andi
  Peng, Mikhail Yurochkin, Atoosa Kasirzadeh, Yangsibo Huang, Tatsunori
  Hashimoto, Yacine Jernite, Daniel Vila-Suero, Omri Abend, Jennifer Ding, Sara
  Hooker, Hannah Rose Kirk, Leshem Choshen},
    journal={arXiv preprint arXiv:2408.16961},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.16961},
    primaryClass={cs.HC cs.AI}
}