PERSONA: A Reproducible Testbed for Pluralistic Alignment
The rapid advancement of language models (LMs) necessitates robust alignment
with diverse user values. However, current preference optimization approaches
often fail to capture the plurality of user opinions, instead reinforcing
majority viewpoints and marginalizing minority perspectives. We introduce
PERSONA, a reproducible test bed designed to evaluate and improve pluralistic
alignment of LMs. We procedurally generate diverse user profiles from US census
data, resulting in 1,586 synthetic personas with varied demographic and
idiosyncratic attributes. We then generate a large-scale evaluation dataset
containing 3,868 prompts and 317,200 feedback pairs obtained from our synthetic
personas. Leveraging this dataset, we systematically evaluate LM capabilities
in role-playing diverse users, verified through human judges, and the
establishment of both a benchmark, PERSONA Bench, for pluralistic alignment
approaches as well as an extensive dataset to create new and future benchmarks.
The full dataset and benchmarks are available here:
https://www.synthlabs.ai/research/persona.
66a1c18c345b3106f46a7fb4
Louis Castricato
admin_assigned
2024-07-25T12:21:47.740Z
60d35a0307da9c17c7270909
https://cdn-avatars.huggingface.co/v1/production/uploads/1665155571839-60d35a0307da9c17c7270909.jpeg
Louis Castricato
user
LouisCastricato
66a1c18c345b3106f46a7fb5
Nathan Lile
admin_assigned
2024-07-25T12:21:54.067Z
61aa15fd8a9625ebfe284286
https://cdn-avatars.huggingface.co/v1/production/uploads/61aa15fd8a9625ebfe284286/KaGzIeijcgcN15JErCqft.jpeg
nathan lile
user
nlile
66a1c18c345b3106f46a7fb6
Rafael Rafailov
admin_assigned
2024-07-25T12:22:00.797Z
64701c98d742e9ef6522598f
/avatars/df3a52bd4bc374d7cb4061f449f75c07.svg
Rafael Rafailov
user
rmrafailov
66a1c18c345b3106f46a7fb7
Jan-Philipp Fränken
admin_assigned
2024-07-25T12:22:05.992Z
63f3b441a096536aeab002b1
/avatars/e5e63b589d791a78e5f1b09b106287bf.svg
Jan-Philipp Fränken
user
janphilippfranken
66a1c18c345b3106f46a7fb8
Chelsea Finn
admin_assigned
2024-07-25T12:22:11.482Z
64ac22a9193f0a807deb673d
/avatars/fcac4912678ad3cb6e817d40bdee9aea.svg
Chelsea Finn
user
cbfinn
PERSONA: A Reproducible Testbed for Pluralistic Alignment
The rapid advancement of language models (LMs) necessitates robust alignment
with diverse user values. However, current preference optimization approaches
often fail to capture the plurality of user opinions, instead reinforcing
majority viewpoints and marginalizing minority perspectives. We introduce
PERSONA, a reproducible test bed designed to evaluate and improve pluralistic
alignment of LMs. We procedurally generate diverse user profiles from US census
data, resulting in 1,586 synthetic personas with varied demographic and
idiosyncratic attributes. We then generate a large-scale evaluation dataset
containing 3,868 prompts and 317,200 feedback pairs obtained from our synthetic
personas. Leveraging this dataset, we systematically evaluate LM capabilities
in role-playing diverse users, verified through human judges, and the
establishment of both a benchmark, PERSONA Bench, for pluralistic alignment
approaches as well as an extensive dataset to create new and future benchmarks.
The full dataset and benchmarks are available here:
https://www.synthlabs.ai/research/persona.
PERSONA is a reproducible test bed that evaluates and enhances the pluralistic alignment of language models using procedurally generated synthetic personas and a large-scale evaluation dataset.
language models preference optimization pluralistic alignment synthetic personas demographic attributes idiosyncratic attributes evaluation dataset prompts feedback pairs human judges benchmark PERSONA Bench