Societal Alignment Frameworks Can Improve LLM Alignment
Recent progress in large language models (LLMs) has focused on producing
responses that meet human expectations and align with shared values - a process
coined alignment. However, aligning LLMs remains challenging due to the
inherent disconnect between the complexity of human values and the narrow
nature of the technological approaches designed to address them. Current
alignment methods often lead to misspecified objectives, reflecting the broader
issue of incomplete contracts, the impracticality of specifying a contract
between a model developer, and the model that accounts for every scenario in
LLM alignment. In this paper, we argue that improving LLM alignment requires
incorporating insights from societal alignment frameworks, including social,
economic, and contractual alignment, and discuss potential solutions drawn from
these domains. Given the role of uncertainty within societal alignment
frameworks, we then investigate how it manifests in LLM alignment. We end our
discussion by offering an alternative view on LLM alignment, framing the
underspecified nature of its objectives as an opportunity rather than perfect
their specification. Beyond technical improvements in LLM alignment, we discuss
the need for participatory alignment interface designs.
67c89f164e249cdfb8b7297d
Karolina Stańczak
extracted_confirmed
2025-03-05T22:06:13.315Z
60a66731e1db8bc33b8d4112
https://cdn-avatars.huggingface.co/v1/production/uploads/60a66731e1db8bc33b8d4112/AY5Y0CnHh08u6lfEoQ6se.jpeg
Karolina Stanczak
user
Karolina
67c89f164e249cdfb8b7297e
Nicholas Meade
67c89f164e249cdfb8b7297f
Mehar Bhatia
claimed_verified
2025-04-13T19:27:42.578Z
63f0546df1a47aaea5bcbae1
/avatars/9d708d88574cea2f17af37b659ef6a53.svg
Mehar Bhatia
user
MeharBhatia
67c89f164e249cdfb8b72980
Hattie Zhou
67c89f164e249cdfb8b72981
Konstantin Böttinger
67c89f164e249cdfb8b72982
Jeremy Barnes
67c89f164e249cdfb8b72983
Jason Stanley
67c89f164e249cdfb8b72984
Jessica Montgomery
67c89f164e249cdfb8b72985
Richard Zemel
67c89f164e249cdfb8b72986
Nicolas Papernot
67c89f164e249cdfb8b72987
Nicolas Chapados
67c89f164e249cdfb8b72988
Denis Therien
67c89f164e249cdfb8b72989
Timothy P. Lillicrap
67c89f164e249cdfb8b7298a
Ana Marasović
67c89f164e249cdfb8b7298b
Sylvie Delacroix
67c89f164e249cdfb8b7298c
Gillian K. Hadfield
67c89f164e249cdfb8b7298d
Siva Reddy
Societal Alignment Frameworks Can Improve LLM Alignment
Recent progress in large language models (LLMs) has focused on producing
responses that meet human expectations and align with shared values - a process
coined alignment. However, aligning LLMs remains challenging due to the
inherent disconnect between the complexity of human values and the narrow
nature of the technological approaches designed to address them. Current
alignment methods often lead to misspecified objectives, reflecting the broader
issue of incomplete contracts, the impracticality of specifying a contract
between a model developer, and the model that accounts for every scenario in
LLM alignment. In this paper, we argue that improving LLM alignment requires
incorporating insights from societal alignment frameworks, including social,
economic, and contractual alignment, and discuss potential solutions drawn from
these domains. Given the role of uncertainty within societal alignment
frameworks, we then investigate how it manifests in LLM alignment. We end our
discussion by offering an alternative view on LLM alignment, framing the
underspecified nature of its objectives as an opportunity rather than perfect
their specification. Beyond technical improvements in LLM alignment, we discuss
the need for participatory alignment interface designs.
Improving large language model alignment requires incorporating insights from societal, economic, and contractual alignment frameworks to address the underspecified nature of alignment objectives.