On the Societal Impact of Open Foundation Models
Foundation models are powerful technologies: how they are released publicly
directly shapes their societal impact. In this position paper, we focus on open
foundation models, defined here as those with broadly available model weights
(e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties
(e.g. greater customizability, poor monitoring) of open foundation models that
lead to both their benefits and risks. Open foundation models present
significant benefits, with some caveats, that span innovation, competition, the
distribution of decision-making power, and transparency. To understand their
risks of misuse, we design a risk assessment framework for analyzing their
marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons),
we find that current research is insufficient to effectively characterize the
marginal risk of open foundation models relative to pre-existing technologies.
The framework helps explain why the marginal risk is low in some cases,
clarifies disagreements about misuse risks by revealing that past work has
focused on different subsets of the framework with different assumptions, and
articulates a way forward for more constructive debate. Overall, our work helps
support a more grounded assessment of the societal impact of open foundation
models by outlining what research is needed to empirically validate their
theoretical benefits and risks.
65f27145915e1f9c232814aa
Sayash Kapoor
extracted_pending
2024-03-14T03:38:45.973Z
630906fde3246e2be88bfab3
/avatars/bca882c027d17f7feba837baae71ec2d.svg
Sayash Kapoor
user
sayashk
65f27145915e1f9c232814ab
Rishi Bommasani
65f27145915e1f9c232814ac
Kevin Klyman
admin_assigned
2024-03-14T08:42:31.827Z
64817160ec65b8b77d83e910
/avatars/0dc892cccb8c71a1ad1c1a1801ee9772.svg
Kevin Klyman
user
kklyman
65f27145915e1f9c232814ad
Shayne Longpre
admin_assigned
2024-03-14T08:42:42.969Z
61f4283a81c4d30f58140242
/avatars/a1cf1ef1fd442c36ed65c68e51919fed.svg
Shayne Longpre
user
Shayne
65f27145915e1f9c232814ae
Ashwin Ramaswami
admin_assigned
2024-03-14T08:42:51.354Z
632a6004ab49d487dd18c53a
/avatars/2150e2452ec5f2fd47ac96488c130e0a.svg
Ashwin Ramaswami
user
epicfaace
65f27145915e1f9c232814af
Peter Cihon
65f27145915e1f9c232814b0
Aspen Hopkins
65f27145915e1f9c232814b1
Kevin Bankston
65f27145915e1f9c232814b2
Stella Biderman
admin_assigned
2024-03-14T08:43:09.947Z
60347d3660e3dd96631c9093
https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
Stella Biderman
user
stellaathena
65f27145915e1f9c232814b3
Miranda Bogen
admin_assigned
2024-03-14T08:43:17.815Z
64becbb0891751ce9b459afb
/avatars/9336cfb09026ef06e0cfa367a7668def.svg
Miranda Bogen
user
mbogen
65f27145915e1f9c232814b4
Rumman Chowdhury
admin_assigned
2024-03-14T08:43:23.631Z
65f0b51065610aef6440de8f
/avatars/ce972a5cf92446e34d8fa9676dc4e23f.svg
Rumman Chowdhury
user
rumman-c
65f27145915e1f9c232814b5
Alex Engler
65f27145915e1f9c232814b6
Peter Henderson
admin_assigned
2024-03-14T08:43:48.259Z
617aafa1ff3db6021d069787
https://cdn-avatars.huggingface.co/v1/production/uploads/1644543485859-617aafa1ff3db6021d069787.jpeg
Peter Henderson
user
breakend
65f27145915e1f9c232814b7
Yacine Jernite
admin_assigned
2024-03-14T08:43:54.857Z
5ee3a7cd2a3eae3cbdad1305
https://cdn-avatars.huggingface.co/v1/production/uploads/1594144055859-5ee3a7cd2a3eae3cbdad1305.jpeg
Yacine Jernite
user
yjernite
65f27145915e1f9c232814b8
Seth Lazar
admin_assigned
2024-03-14T08:44:04.436Z
631861b8e5a3818041720b12
/avatars/f9a017fb46626b0c143ff18bd5fa37b7.svg
Seth Lazar
user
sethlazar
65f27145915e1f9c232814b9
Stefano Maffulli
65f27145915e1f9c232814ba
Alondra Nelson
65f27145915e1f9c232814bb
Joelle Pineau
65f27145915e1f9c232814bc
Aviya Skowron
admin_assigned
2024-03-14T08:44:28.156Z
63c5dfc8d5a5cd2043e6f03c
/avatars/edcfcd9cfb03286d670e6c5743efef6a.svg
Aviya Skowron
user
avi-skowron
65f27145915e1f9c232814bd
Dawn Song
admin_assigned
2024-03-14T08:44:34.514Z
6513179af60393414afd4140
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/GAf-FulRRNF-X_Svew1Xj.jpeg
Xiaowei Song
user
dawnsong
65f27145915e1f9c232814be
Victor Storchan
65f27145915e1f9c232814bf
Daniel Zhang
65f27145915e1f9c232814c0
Daniel E. Ho
65f27145915e1f9c232814c1
Percy Liang
admin_assigned
2024-03-14T08:45:04.664Z
6409651b9e9f790c905b2335
/avatars/1fb8c80b60f21f65a0a027319101f236.svg
Percy Liang
user
percyliang
65f27145915e1f9c232814c2
Arvind Narayanan
On the Societal Impact of Open Foundation Models
Foundation models are powerful technologies: how they are released publicly
directly shapes their societal impact. In this position paper, we focus on open
foundation models, defined here as those with broadly available model weights
(e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties
(e.g. greater customizability, poor monitoring) of open foundation models that
lead to both their benefits and risks. Open foundation models present
significant benefits, with some caveats, that span innovation, competition, the
distribution of decision-making power, and transparency. To understand their
risks of misuse, we design a risk assessment framework for analyzing their
marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons),
we find that current research is insufficient to effectively characterize the
marginal risk of open foundation models relative to pre-existing technologies.
The framework helps explain why the marginal risk is low in some cases,
clarifies disagreements about misuse risks by revealing that past work has
focused on different subsets of the framework with different assumptions, and
articulates a way forward for more constructive debate. Overall, our work helps
support a more grounded assessment of the societal impact of open foundation
models by outlining what research is needed to empirically validate their
theoretical benefits and risks.