title: From Pixels to Prose: A Large Dataset of Dense Image Captions
summary: Training large vision-language models requires extensive, high-quality
  image-text pairs. Existing web-scraped datasets, however, are noisy and lack
  detailed image descriptions. To bridge this gap, we introduce PixelProse, a
  comprehensive dataset of over 16M (million) synthetically generated captions,
  leveraging cutting-edge vision-language models for detailed and accurate
  descriptions. To ensure data integrity, we rigorously analyze our dataset for
  problematic content, including child sexual abuse material (CSAM), personally
  identifiable information (PII), and toxicity. We also provide valuable metadata
  such as watermark presence and aesthetic scores, aiding in further dataset
  filtering. We hope PixelProse will be a valuable resource for future
  vision-language research. PixelProse is available at
  https://huggingface.co/datasets/tomg-group-umd/pixelprose
paper_authors._id: 6670ff4b244fc188ec2ee635
paper_authors.name: Vasu Singla
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-11T14:19:03.578Z
paper_authors._id: 6352f7f90422304c3e7bf098
paper_authors.avatarUrl: /avatars/90bff66d85240415bb43d02c329ffbca.svg
paper_authors.fullname: Vasu Singla
paper_authors.type: user
paper_authors.user: vasusingla
paper_authors._id: 6670ff4b244fc188ec2ee636
paper_authors.name: Kaiyu Yue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-20T07:34:40.773Z
paper_authors._id: 640d0dbc8036cc2142273a83
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640d0dbc8036cc2142273a83/TQHyWWMGuXJKEM6fTPf-z.png
paper_authors.fullname: Kaiyu Yue
paper_authors.type: user
paper_authors.user: kaiyuyue
paper_authors._id: 6670ff4b244fc188ec2ee637
paper_authors.name: Sukriti Paul
paper_authors._id: 6670ff4b244fc188ec2ee638
paper_authors.name: Reza Shirkavand
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-19T20:21:14.265Z
paper_authors._id: 65466e8b4d1931dc93dbf9a6
paper_authors.avatarUrl: /avatars/8626417bac65f431bc50345a024dd570.svg
paper_authors.fullname: Reza Shirkavand
paper_authors.type: user
paper_authors.user: rezashkv
paper_authors._id: 6670ff4b244fc188ec2ee639
paper_authors.name: Mayuka Jayawardhana
paper_authors._id: 6670ff4b244fc188ec2ee63a
paper_authors.name: Alireza Ganjdanesh
paper_authors._id: 6670ff4b244fc188ec2ee63b
paper_authors.name: Heng Huang
paper_authors._id: 6670ff4b244fc188ec2ee63c
paper_authors.name: Abhinav Bhatele
paper_authors._id: 6670ff4b244fc188ec2ee63d
paper_authors.name: Gowthami Somepalli
paper_authors._id: 6670ff4b244fc188ec2ee63e
paper_authors.name: Tom Goldstein
paper_title: From Pixels to Prose: A Large Dataset of Dense Image Captions
paper_summary: Training large vision-language models requires extensive, high-quality
  image-text pairs. Existing web-scraped datasets, however, are noisy and lack
  detailed image descriptions. To bridge this gap, we introduce PixelProse, a
  comprehensive dataset of over 16M (million) synthetically generated captions,
  leveraging cutting-edge vision-language models for detailed and accurate
  descriptions. To ensure data integrity, we rigorously analyze our dataset for
  problematic content, including child sexual abuse material (CSAM), personally
  identifiable information (PII), and toxicity. We also provide valuable metadata
  such as watermark presence and aesthetic scores, aiding in further dataset
  filtering. We hope PixelProse will be a valuable resource for future
  vision-language research. PixelProse is available at
  https://huggingface.co/datasets/tomg-group-umd/pixelprose
paper_ai_keywords: vision-language models image-text pairs web-scraped datasets synthetically generated captions data integrity child sexual abuse material personally identifiable information toxicity metadata watermark presence aesthetic scores dataset filtering