title: OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text
    Documents
summary: Large multimodal models trained on natural documents, which interleave images
  and text, outperform models trained on image-text pairs on various multimodal
  benchmarks. However, the datasets used to train these models have not been
  released, and the collection process has not been fully specified. We introduce
  the OBELICS dataset, an open web-scale filtered dataset of interleaved
  image-text documents comprising 141 million web pages extracted from Common
  Crawl, 353 million associated images, and 115 billion text tokens. We describe
  the dataset creation process, present comprehensive filtering rules, and
  provide an analysis of the dataset's content. To show the viability of OBELICS,
  we train vision and language models of 9 and 80 billion parameters named
  IDEFICS, and obtain competitive performance on different multimodal benchmarks.
  We release our dataset, models and code.
paper_authors._id: 649e28bdba50848bd6c59e4f
paper_authors.name: Hugo Laurençon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-30T16:17:31.942Z
paper_authors._id: 6177322d37f32ecb1e2d4cdf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1635201569275-noauth.jpeg
paper_authors.fullname: Hugo Laurençon
paper_authors.type: user
paper_authors.user: HugoLaurencon
paper_authors._id: 649e28bdba50848bd6c59e50
paper_authors.name: Lucile Saulnier
paper_authors._id: 649e28bdba50848bd6c59e51
paper_authors.name: Léo Tronchon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-30T10:23:16.478Z
paper_authors._id: 6244866a456803e9500d0f6a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652185658647-6244866a456803e9500d0f6a.jpeg
paper_authors.fullname: Leo Tronchon
paper_authors.type: user
paper_authors.user: Leyo
paper_authors._id: 649e28bdba50848bd6c59e52
paper_authors.name: Stas Bekman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-28T15:08:06.714Z
paper_authors._id: 5f07383b19cb630495b812cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1594311341799-5f07383b19cb630495b812cd.jpeg
paper_authors.fullname: Stas Bekman
paper_authors.type: user
paper_authors.user: stas
paper_authors._id: 649e28bdba50848bd6c59e53
paper_authors.name: Amanpreet Singh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-19T09:36:05.941Z
paper_authors._id: 6230c6ecfd8b720a5648f6c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652253065548-6230c6ecfd8b720a5648f6c4.jpeg
paper_authors.fullname: Amanpreet Singh
paper_authors.type: user
paper_authors.user: aps
paper_authors._id: 649e28bdba50848bd6c59e54
paper_authors.name: Anton Lozhkov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-30T16:17:34.987Z
paper_authors._id: 602e6dee60e3dd96631c906e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1613655355830-noauth.png
paper_authors.fullname: Anton Lozhkov
paper_authors.type: user
paper_authors.user: anton-l
paper_authors._id: 649e28bdba50848bd6c59e55
paper_authors.name: Thomas Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-11T18:41:56.710Z
paper_authors._id: 61140e35d08630d2676c9829
paper_authors.avatarUrl: /avatars/3d6e4b4d02eda7b5ef28e1cc0fb8e08a.svg
paper_authors.fullname: Thomas Wang
paper_authors.type: user
paper_authors.user: TimeRobber
paper_authors._id: 649e28bdba50848bd6c59e56
paper_authors.name: Siddharth Karamcheti
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-15T00:35:38.943Z
paper_authors._id: 6150b090d84cf0532aa1764b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1634666324094-6150b090d84cf0532aa1764b.jpeg
paper_authors.fullname: Siddharth Karamcheti
paper_authors.type: user
paper_authors.user: skaramcheti
paper_authors._id: 649e28bdba50848bd6c59e57
paper_authors.name: Alexander M. Rush
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T09:52:19.798Z
paper_authors._id: 5e694f697127cb58dde42fe1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/J21QWWtFOKqhJTF5qIMlD.jpeg
paper_authors.fullname: Sasha Rush
paper_authors.type: user
paper_authors.user: srush
paper_authors._id: 649e28bdba50848bd6c59e58
paper_authors.name: Douwe Kiela
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T09:52:31.092Z
paper_authors._id: 61dc997715b47073db1620dc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1641847245435-61dc997715b47073db1620dc.jpeg
paper_authors.fullname: Douwe Kiela
paper_authors.type: user
paper_authors.user: douwekiela
paper_authors._id: 649e28bdba50848bd6c59e59
paper_authors.name: Matthieu Cord
paper_authors._id: 649e28bdba50848bd6c59e5a
paper_authors.name: Victor Sanh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T09:52:03.972Z
paper_authors._id: 5ecea265968f6028e0559fa5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1619623771844-5ecea265968f6028e0559fa5.jpeg
paper_authors.fullname: Victor Sanh
paper_authors.type: user
paper_authors.user: VictorSanh
paper_title: OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text
    Documents
paper_summary: Large multimodal models trained on natural documents, which interleave images
  and text, outperform models trained on image-text pairs on various multimodal
  benchmarks. However, the datasets used to train these models have not been
  released, and the collection process has not been fully specified. We introduce
  the OBELICS dataset, an open web-scale filtered dataset of interleaved
  image-text documents comprising 141 million web pages extracted from Common
  Crawl, 353 million associated images, and 115 billion text tokens. We describe
  the dataset creation process, present comprehensive filtering rules, and
  provide an analysis of the dataset's content. To show the viability of OBELICS,
  we train vision and language models of 9 and 80 billion parameters named
  IDEFICS, and obtain competitive performance on different multimodal benchmarks.
  We release our dataset, models and code.
paper_ai_summary: The OBELICS dataset, containing interleaved image-text documents, is introduced and used to train competitive multimodal models.
paper_ai_keywords: multimodal models natural documents interleave images image-text pairs multimodal benchmarks OBELICS dataset Common Crawl vision and language models IDEFICS