title: On the Societal Impact of Open Foundation Models
summary: Foundation models are powerful technologies: how they are released publicly
  directly shapes their societal impact. In this position paper, we focus on open
  foundation models, defined here as those with broadly available model weights
  (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties
  (e.g. greater customizability, poor monitoring) of open foundation models that
  lead to both their benefits and risks. Open foundation models present
  significant benefits, with some caveats, that span innovation, competition, the
  distribution of decision-making power, and transparency. To understand their
  risks of misuse, we design a risk assessment framework for analyzing their
  marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons),
  we find that current research is insufficient to effectively characterize the
  marginal risk of open foundation models relative to pre-existing technologies.
  The framework helps explain why the marginal risk is low in some cases,
  clarifies disagreements about misuse risks by revealing that past work has
  focused on different subsets of the framework with different assumptions, and
  articulates a way forward for more constructive debate. Overall, our work helps
  support a more grounded assessment of the societal impact of open foundation
  models by outlining what research is needed to empirically validate their
  theoretical benefits and risks.
paper_authors._id: 65f27145915e1f9c232814aa
paper_authors.name: Sayash Kapoor
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-03-14T03:38:45.973Z
paper_authors._id: 630906fde3246e2be88bfab3
paper_authors.avatarUrl: /avatars/bca882c027d17f7feba837baae71ec2d.svg
paper_authors.fullname: Sayash Kapoor
paper_authors.type: user
paper_authors.user: sayashk
paper_authors._id: 65f27145915e1f9c232814ab
paper_authors.name: Rishi Bommasani
paper_authors._id: 65f27145915e1f9c232814ac
paper_authors.name: Kevin Klyman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:42:31.827Z
paper_authors._id: 64817160ec65b8b77d83e910
paper_authors.avatarUrl: /avatars/0dc892cccb8c71a1ad1c1a1801ee9772.svg
paper_authors.fullname: Kevin Klyman
paper_authors.type: user
paper_authors.user: kklyman
paper_authors._id: 65f27145915e1f9c232814ad
paper_authors.name: Shayne Longpre
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:42:42.969Z
paper_authors._id: 61f4283a81c4d30f58140242
paper_authors.avatarUrl: /avatars/a1cf1ef1fd442c36ed65c68e51919fed.svg
paper_authors.fullname: Shayne Longpre
paper_authors.type: user
paper_authors.user: Shayne
paper_authors._id: 65f27145915e1f9c232814ae
paper_authors.name: Ashwin Ramaswami
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:42:51.354Z
paper_authors._id: 632a6004ab49d487dd18c53a
paper_authors.avatarUrl: /avatars/2150e2452ec5f2fd47ac96488c130e0a.svg
paper_authors.fullname: Ashwin Ramaswami
paper_authors.type: user
paper_authors.user: epicfaace
paper_authors._id: 65f27145915e1f9c232814af
paper_authors.name: Peter Cihon
paper_authors._id: 65f27145915e1f9c232814b0
paper_authors.name: Aspen Hopkins
paper_authors._id: 65f27145915e1f9c232814b1
paper_authors.name: Kevin Bankston
paper_authors._id: 65f27145915e1f9c232814b2
paper_authors.name: Stella Biderman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:43:09.947Z
paper_authors._id: 60347d3660e3dd96631c9093
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
paper_authors.fullname: Stella Biderman
paper_authors.type: user
paper_authors.user: stellaathena
paper_authors._id: 65f27145915e1f9c232814b3
paper_authors.name: Miranda Bogen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:43:17.815Z
paper_authors._id: 64becbb0891751ce9b459afb
paper_authors.avatarUrl: /avatars/9336cfb09026ef06e0cfa367a7668def.svg
paper_authors.fullname: Miranda Bogen
paper_authors.type: user
paper_authors.user: mbogen
paper_authors._id: 65f27145915e1f9c232814b4
paper_authors.name: Rumman Chowdhury
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:43:23.631Z
paper_authors._id: 65f0b51065610aef6440de8f
paper_authors.avatarUrl: /avatars/ce972a5cf92446e34d8fa9676dc4e23f.svg
paper_authors.fullname: Rumman Chowdhury
paper_authors.type: user
paper_authors.user: rumman-c
paper_authors._id: 65f27145915e1f9c232814b5
paper_authors.name: Alex Engler
paper_authors._id: 65f27145915e1f9c232814b6
paper_authors.name: Peter Henderson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:43:48.259Z
paper_authors._id: 617aafa1ff3db6021d069787
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644543485859-617aafa1ff3db6021d069787.jpeg
paper_authors.fullname: Peter Henderson
paper_authors.type: user
paper_authors.user: breakend
paper_authors._id: 65f27145915e1f9c232814b7
paper_authors.name: Yacine Jernite
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:43:54.857Z
paper_authors._id: 5ee3a7cd2a3eae3cbdad1305
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1594144055859-5ee3a7cd2a3eae3cbdad1305.jpeg
paper_authors.fullname: Yacine Jernite
paper_authors.type: user
paper_authors.user: yjernite
paper_authors._id: 65f27145915e1f9c232814b8
paper_authors.name: Seth Lazar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:44:04.436Z
paper_authors._id: 631861b8e5a3818041720b12
paper_authors.avatarUrl: /avatars/f9a017fb46626b0c143ff18bd5fa37b7.svg
paper_authors.fullname: Seth Lazar
paper_authors.type: user
paper_authors.user: sethlazar
paper_authors._id: 65f27145915e1f9c232814b9
paper_authors.name: Stefano Maffulli
paper_authors._id: 65f27145915e1f9c232814ba
paper_authors.name: Alondra Nelson
paper_authors._id: 65f27145915e1f9c232814bb
paper_authors.name: Joelle Pineau
paper_authors._id: 65f27145915e1f9c232814bc
paper_authors.name: Aviya Skowron
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:44:28.156Z
paper_authors._id: 63c5dfc8d5a5cd2043e6f03c
paper_authors.avatarUrl: /avatars/edcfcd9cfb03286d670e6c5743efef6a.svg
paper_authors.fullname: Aviya Skowron
paper_authors.type: user
paper_authors.user: avi-skowron
paper_authors._id: 65f27145915e1f9c232814bd
paper_authors.name: Dawn Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:44:34.514Z
paper_authors._id: 6513179af60393414afd4140
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/GAf-FulRRNF-X_Svew1Xj.jpeg
paper_authors.fullname: Xiaowei Song
paper_authors.type: user
paper_authors.user: dawnsong
paper_authors._id: 65f27145915e1f9c232814be
paper_authors.name: Victor Storchan
paper_authors._id: 65f27145915e1f9c232814bf
paper_authors.name: Daniel Zhang
paper_authors._id: 65f27145915e1f9c232814c0
paper_authors.name: Daniel E. Ho
paper_authors._id: 65f27145915e1f9c232814c1
paper_authors.name: Percy Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-14T08:45:04.664Z
paper_authors._id: 6409651b9e9f790c905b2335
paper_authors.avatarUrl: /avatars/1fb8c80b60f21f65a0a027319101f236.svg
paper_authors.fullname: Percy Liang
paper_authors.type: user
paper_authors.user: percyliang
paper_authors._id: 65f27145915e1f9c232814c2
paper_authors.name: Arvind Narayanan
paper_title: On the Societal Impact of Open Foundation Models
paper_summary: Foundation models are powerful technologies: how they are released publicly
  directly shapes their societal impact. In this position paper, we focus on open
  foundation models, defined here as those with broadly available model weights
  (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties
  (e.g. greater customizability, poor monitoring) of open foundation models that
  lead to both their benefits and risks. Open foundation models present
  significant benefits, with some caveats, that span innovation, competition, the
  distribution of decision-making power, and transparency. To understand their
  risks of misuse, we design a risk assessment framework for analyzing their
  marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons),
  we find that current research is insufficient to effectively characterize the
  marginal risk of open foundation models relative to pre-existing technologies.
  The framework helps explain why the marginal risk is low in some cases,
  clarifies disagreements about misuse risks by revealing that past work has
  focused on different subsets of the framework with different assumptions, and
  articulates a way forward for more constructive debate. Overall, our work helps
  support a more grounded assessment of the societal impact of open foundation
  models by outlining what research is needed to empirically validate their
  theoretical benefits and risks.