title: Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual
    Concept Understanding
summary: Vision and vision-language applications of neural networks, such as image
  classification and captioning, rely on large-scale annotated datasets that
  require non-trivial data-collecting processes. This time-consuming endeavor
  hinders the emergence of large-scale datasets, limiting researchers and
  practitioners to a small number of choices. Therefore, we seek more efficient
  ways to collect and annotate images. Previous initiatives have gathered
  captions from HTML alt-texts and crawled social media postings, but these data
  sources suffer from noise, sparsity, or subjectivity. For this reason, we turn
  to commercial shopping websites whose data meet three criteria: cleanliness,
  informativeness, and fluency. We introduce the Let's Go Shopping (LGS) dataset,
  a large-scale public dataset with 15 million image-caption pairs from publicly
  available e-commerce websites. When compared with existing general-domain
  datasets, the LGS images focus on the foreground object and have less complex
  backgrounds. Our experiments on LGS show that the classifiers trained on
  existing benchmark datasets do not readily generalize to e-commerce data, while
  specific self-supervised visual feature extractors can better generalize.
  Furthermore, LGS's high-quality e-commerce-focused images and bimodal nature
  make it advantageous for vision-language bi-modal tasks: LGS enables
  image-captioning models to generate richer captions and helps text-to-image
  generation models achieve e-commerce style transfer.
paper_authors._id: 659e0d65e6fe8ef6195521f4
paper_authors.name: Yatong Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:06:25.655Z
paper_authors._id: 63239bd492e07e3ca2068a16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63239bd492e07e3ca2068a16/OFESFFWQVHC1EtJpnGOWv.jpeg
paper_authors.fullname: Yatong Bai
paper_authors.type: user
paper_authors.user: Bai-YT
paper_authors._id: 659e0d65e6fe8ef6195521f5
paper_authors.name: Utsav Garg
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:06:32.495Z
paper_authors._id: 631fd216c1a8269da399e249
paper_authors.avatarUrl: /avatars/5334e8be63fc43bd3351af4fe5935e69.svg
paper_authors.fullname: Utsav Garg
paper_authors.type: user
paper_authors.user: utsavgarg
paper_authors._id: 659e0d65e6fe8ef6195521f6
paper_authors.name: Apaar Shanker
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:06:38.711Z
paper_authors._id: 6309949703837fbcf0860288
paper_authors.avatarUrl: /avatars/0ef388cf224bfedde279f5cb2b7bf592.svg
paper_authors.fullname: Apaar Shanker
paper_authors.type: user
paper_authors.user: ashanker9
paper_authors._id: 659e0d65e6fe8ef6195521f7
paper_authors.name: Haoming Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:06:56.015Z
paper_authors._id: 64ead129f6d7c8bdfb99593d
paper_authors.avatarUrl: /avatars/5c657d72ef6c59f182b2c5113f1c0e79.svg
paper_authors.fullname: ZhangHaoMing
paper_authors.type: user
paper_authors.user: Blackfirezhm
paper_authors._id: 659e0d65e6fe8ef6195521f8
paper_authors.name: Samyak Parajuli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:07:04.605Z
paper_authors._id: 631ba8448de8e645af70820c
paper_authors.avatarUrl: /avatars/ff5709d81010d8980f637775d92a71a5.svg
paper_authors.fullname: Samyak Parajuli
paper_authors.type: user
paper_authors.user: samp830
paper_authors._id: 659e0d65e6fe8ef6195521f9
paper_authors.name: Erhan Bas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:18:55.737Z
paper_authors._id: 635051ba4a9fc01602f7f87a
paper_authors.avatarUrl: /avatars/f22fd2f02d4163f0a620727d9b79cc4a.svg
paper_authors.fullname: Erhan
paper_authors.type: user
paper_authors.user: erhanbas
paper_authors._id: 659e0d65e6fe8ef6195521fa
paper_authors.name: Isidora Filipovic
paper_authors._id: 659e0d65e6fe8ef6195521fb
paper_authors.name: Amelia N. Chu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:19:30.619Z
paper_authors._id: 63470824e46b06a94a7fbb78
paper_authors.avatarUrl: /avatars/44d22b8b6af085cf4fa2dec338232f2a.svg
paper_authors.fullname: Amelia Chu
paper_authors.type: user
paper_authors.user: ameliachu
paper_authors._id: 659e0d65e6fe8ef6195521fc
paper_authors.name: Eugenia D Fomitcheva
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-02T19:14:22.205Z
paper_authors._id: 63757945885ea5d19f8cb2a6
paper_authors.avatarUrl: /avatars/e933d6e1db86552492ec651cfacc1179.svg
paper_authors.fullname: Eug Fomitcheva
paper_authors.type: user
paper_authors.user: eugfomitcheva
paper_authors._id: 659e0d65e6fe8ef6195521fd
paper_authors.name: Elliot Branson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:19:46.659Z
paper_authors._id: 627ade6576a15e23898ae25a
paper_authors.avatarUrl: /avatars/0ec3943018237bf407373030f433bea4.svg
paper_authors.fullname: Elliot Branson
paper_authors.type: user
paper_authors.user: ElliotBranson
paper_authors._id: 659e0d65e6fe8ef6195521fe
paper_authors.name: Aerin Kim
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-01-10T03:27:05.979Z
paper_authors._id: 630629d9660f01f150a110d3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630629d9660f01f150a110d3/-9_cmjec1aEdX_1BKosCk.png
paper_authors.fullname: aerin kim
paper_authors.type: user
paper_authors.user: aerinkim
paper_authors._id: 659e0d65e6fe8ef6195521ff
paper_authors.name: Somayeh Sojoudi
paper_authors._id: 659e0d65e6fe8ef619552200
paper_authors.name: Kyunghyun Cho
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-10T09:20:04.684Z
paper_authors._id: 649c1b8485b05ee02723334a
paper_authors.avatarUrl: /avatars/29080f550fdb471c4d2daace86aff3b0.svg
paper_authors.fullname: Kyunghyun Cho
paper_authors.type: user
paper_authors.user: kyunghyuncho
paper_title: Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual
    Concept Understanding
paper_summary: Vision and vision-language applications of neural networks, such as image
  classification and captioning, rely on large-scale annotated datasets that
  require non-trivial data-collecting processes. This time-consuming endeavor
  hinders the emergence of large-scale datasets, limiting researchers and
  practitioners to a small number of choices. Therefore, we seek more efficient
  ways to collect and annotate images. Previous initiatives have gathered
  captions from HTML alt-texts and crawled social media postings, but these data
  sources suffer from noise, sparsity, or subjectivity. For this reason, we turn
  to commercial shopping websites whose data meet three criteria: cleanliness,
  informativeness, and fluency. We introduce the Let's Go Shopping (LGS) dataset,
  a large-scale public dataset with 15 million image-caption pairs from publicly
  available e-commerce websites. When compared with existing general-domain
  datasets, the LGS images focus on the foreground object and have less complex
  backgrounds. Our experiments on LGS show that the classifiers trained on
  existing benchmark datasets do not readily generalize to e-commerce data, while
  specific self-supervised visual feature extractors can better generalize.
  Furthermore, LGS's high-quality e-commerce-focused images and bimodal nature
  make it advantageous for vision-language bi-modal tasks: LGS enables
  image-captioning models to generate richer captions and helps text-to-image
  generation models achieve e-commerce style transfer.
paper_ai_keywords: Let's Go Shopping (LGS) dataset image-caption pairs e-commerce websites visual feature extractors image-captioning models text-to-image generation models e-commerce style transfer