BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions
We introduce BLIP3-KALE, a dataset of 218 million image-text pairs that
bridges the gap between descriptive synthetic captions and factual web-scale
alt-text. KALE augments synthetic dense image captions with web-scale alt-text
to generate factually grounded image captions. Our two-stage approach leverages
large vision-language models and language models to create knowledge-augmented
captions, which are then used to train a specialized VLM for scaling up the
dataset. We train vision-language models on KALE and demonstrate improvements
on vision-language tasks. Our experiments show the utility of KALE for training
more capable and knowledgeable multimodal models. We release the KALE dataset
at https://huggingface.co/datasets/Salesforce/blip3-kale
6734bf91fa1009ea2b5a1534
Anas Awadalla
claimed_verified
2024-11-15T09:22:30.456Z
60f1ab3afe0d78e01037eeb1
/avatars/f784fa423fd84fffb4683fa837ffc5a3.svg
Anas Awadalla
user
anas-awadalla
6734bf91fa1009ea2b5a1535
Le Xue
6734bf91fa1009ea2b5a1536
Manli Shu
6734bf91fa1009ea2b5a1537
An Yan
claimed_verified
2024-11-15T09:22:28.494Z
634dfb9bbe5a827d48749f39
/avatars/a8df0eeb1df39f3935dd686a94768f30.svg
An Yan
user
zzxslp
6734bf91fa1009ea2b5a1538
Jun Wang
6734bf91fa1009ea2b5a1539
Senthil Purushwalkam
6734bf91fa1009ea2b5a153a
Sheng Shen
6734bf91fa1009ea2b5a153b
Hannah Lee
6734bf91fa1009ea2b5a153c
Oscar Lo
6734bf91fa1009ea2b5a153d
Jae Sung Park
6734bf91fa1009ea2b5a153e
Etash Guha
6734bf91fa1009ea2b5a153f
Silvio Savarese
6734bf91fa1009ea2b5a1540
Ludwig Schmidt
6734bf91fa1009ea2b5a1541
Yejin Choi
6734bf91fa1009ea2b5a1542
Caiming Xiong
6734bf91fa1009ea2b5a1543
Ran Xu
claimed_verified
2025-05-15T10:41:24.701Z
6465c4c863e7e09dd02e3e1b
/avatars/200b029184d2616f98296a2c212f0785.svg
Ran Xu
user
xurantju
BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions
We introduce BLIP3-KALE, a dataset of 218 million image-text pairs that
bridges the gap between descriptive synthetic captions and factual web-scale
alt-text. KALE augments synthetic dense image captions with web-scale alt-text
to generate factually grounded image captions. Our two-stage approach leverages
large vision-language models and language models to create knowledge-augmented
captions, which are then used to train a specialized VLM for scaling up the
dataset. We train vision-language models on KALE and demonstrate improvements
on vision-language tasks. Our experiments show the utility of KALE for training
more capable and knowledgeable multimodal models. We release the KALE dataset
at https://huggingface.co/datasets/Salesforce/blip3-kale
A dataset of 218 million image-text pairs bridges synthetic and factual captions, enhancing vision-language models.
vision-language models knowledge-augmented captions multimodal models