title: Open-Endedness is Essential for Artificial Superhuman Intelligence
summary: In recent years there has been a tremendous surge in the general capabilities
  of AI systems, mainly fuelled by training foundation models on internetscale
  data. Nevertheless, the creation of openended, ever self-improving AI remains
  elusive. In this position paper, we argue that the ingredients are now in place
  to achieve openendedness in AI systems with respect to a human observer.
  Furthermore, we claim that such open-endedness is an essential property of any
  artificial superhuman intelligence (ASI). We begin by providing a concrete
  formal definition of open-endedness through the lens of novelty and
  learnability. We then illustrate a path towards ASI via open-ended systems
  built on top of foundation models, capable of making novel, humanrelevant
  discoveries. We conclude by examining the safety implications of
  generally-capable openended AI. We expect that open-ended foundation models
  will prove to be an increasingly fertile and safety-critical area of research
  in the near future.
paper_authors._id: 66627275648349b3930079ba
paper_authors.name: Edward Hughes
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-07T12:54:03.783Z
paper_authors._id: 64dcad22249785efad8da97b
paper_authors.avatarUrl: /avatars/11ebe5dec1e5d2e3c7cd00ec684e2319.svg
paper_authors.fullname: Edward Hughes
paper_authors.type: user
paper_authors.user: visible-ed
paper_authors._id: 66627275648349b3930079bb
paper_authors.name: Michael Dennis
paper_authors._id: 66627275648349b3930079bc
paper_authors.name: Jack Parker-Holder
paper_authors._id: 66627275648349b3930079bd
paper_authors.name: Feryal Behbahani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-07T12:54:36.327Z
paper_authors._id: 64c197b7ec3c6181351b2c5d
paper_authors.avatarUrl: /avatars/f81fe98f68720b95885d4968b0e7cc0f.svg
paper_authors.fullname: Feryal Behbahani
paper_authors.type: user
paper_authors.user: ferya
paper_authors._id: 66627275648349b3930079be
paper_authors.name: Aditi Mavalankar
paper_authors._id: 66627275648349b3930079bf
paper_authors.name: Yuge Shi
paper_authors._id: 66627275648349b3930079c0
paper_authors.name: Tom Schaul
paper_authors._id: 66627275648349b3930079c1
paper_authors.name: Tim Rocktaschel
paper_title: Open-Endedness is Essential for Artificial Superhuman Intelligence
paper_summary: In recent years there has been a tremendous surge in the general capabilities
  of AI systems, mainly fuelled by training foundation models on internetscale
  data. Nevertheless, the creation of openended, ever self-improving AI remains
  elusive. In this position paper, we argue that the ingredients are now in place
  to achieve openendedness in AI systems with respect to a human observer.
  Furthermore, we claim that such open-endedness is an essential property of any
  artificial superhuman intelligence (ASI). We begin by providing a concrete
  formal definition of open-endedness through the lens of novelty and
  learnability. We then illustrate a path towards ASI via open-ended systems
  built on top of foundation models, capable of making novel, humanrelevant
  discoveries. We conclude by examining the safety implications of
  generally-capable openended AI. We expect that open-ended foundation models
  will prove to be an increasingly fertile and safety-critical area of research
  in the near future.
paper_ai_keywords: foundational models open-endedness ASI (artificial superhuman intelligence) novelty learnability open-ended systems human-relevant discoveries safety implications