JourneyDB: A Benchmark for Generative Image Understanding
While recent advancements in vision-language models have revolutionized
multi-modal understanding, it remains unclear whether they possess the
capabilities of comprehending the generated images. Compared to real data,
synthetic images exhibit a higher degree of diversity in both content and
style, for which there are significant difficulties for the models to fully
apprehend. To this end, we present a large-scale dataset, JourneyDB, for
multi-modal visual understanding in generative images. Our curated dataset
covers 4 million diverse and high-quality generated images paired with the text
prompts used to produce them. We further design 4 benchmarks to quantify the
performance of generated image understanding in terms of both content and style
interpretation. These benchmarks include prompt inversion, style retrieval,
image captioning and visual question answering. Lastly, we assess the
performance of current state-of-the-art multi-modal models when applied to
JourneyDB, and provide an in-depth analysis of their strengths and limitations
in generated content understanding. We hope the proposed dataset and benchmarks
will facilitate the research in the field of generative content understanding.
The dataset will be available on https://journeydb.github.io.
64a384400d5af4b16211ee2b
Junting Pan
64a384400d5af4b16211ee2c
Keqiang Sun
claimed_verified
2024-11-15T09:55:55.200Z
645ca9f3a19f3e64bbed7534
/avatars/37fde5fd978bc03a3a02c30102557432.svg
Keqiang Sun
user
kqsun
64a384400d5af4b16211ee2d
Yuying Ge
64a384400d5af4b16211ee2e
Hao Li
claimed_verified
2024-12-16T09:44:44.723Z
64b9033777ae61bcc80aa4f3
/avatars/408c335395c79f3df69fd9bf70abc312.svg
Hao Li
user
cpsxhao
64a384400d5af4b16211ee2f
Haodong Duan
claimed_verified
2024-04-24T12:50:36.834Z
63ee1379190ddd6214efd73a
https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
HAODONG DUAN
user
KennyUTC
64a384400d5af4b16211ee30
Xiaoshi Wu
claimed_verified
2023-07-04T19:17:00.843Z
6423f44d30b0e4ab36dd939d
/avatars/9bbf6f00188d21d7bd5b678b955b9fc0.svg
Xiaoshi Wu
user
xswu
64a384400d5af4b16211ee31
Renrui Zhang
claimed_verified
2024-03-22T08:40:57.242Z
645b8b2687c79b6ec0bb3b7a
/avatars/00a9db32a42dc950112bf2593bb109cb.svg
Renrui
user
ZrrSkywalker
64a384400d5af4b16211ee32
Aojun Zhou
claimed_verified
2024-03-22T10:06:08.713Z
637de1520d5bb06fbe5207a9
/avatars/1090851217270c5a858b13e013356d4f.svg
AJ.Zhou
user
AJZhou
64a384400d5af4b16211ee33
Zipeng Qin
64a384400d5af4b16211ee34
Yi Wang
claimed_verified
2023-07-05T18:56:08.991Z
63119a987680dc699b2031df
/avatars/db63ab73806ee9cb8aa01be82d9effdd.svg
Yi Wang
user
shepnerd
64a384400d5af4b16211ee35
Jifeng Dai
64a384400d5af4b16211ee36
Yu Qiao
64a384400d5af4b16211ee37
Hongsheng Li
claimed_verified
2024-02-05T10:50:51.845Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
JourneyDB: A Benchmark for Generative Image Understanding
While recent advancements in vision-language models have revolutionized
multi-modal understanding, it remains unclear whether they possess the
capabilities of comprehending the generated images. Compared to real data,
synthetic images exhibit a higher degree of diversity in both content and
style, for which there are significant difficulties for the models to fully
apprehend. To this end, we present a large-scale dataset, JourneyDB, for
multi-modal visual understanding in generative images. Our curated dataset
covers 4 million diverse and high-quality generated images paired with the text
prompts used to produce them. We further design 4 benchmarks to quantify the
performance of generated image understanding in terms of both content and style
interpretation. These benchmarks include prompt inversion, style retrieval,
image captioning and visual question answering. Lastly, we assess the
performance of current state-of-the-art multi-modal models when applied to
JourneyDB, and provide an in-depth analysis of their strengths and limitations
in generated content understanding. We hope the proposed dataset and benchmarks
will facilitate the research in the field of generative content understanding.
The dataset will be available on https://journeydb.github.io.
A large-scale dataset called JourneyDB with 4 million generated images and text prompts is introduced to evaluate multi-modal models' performance in understanding generated images through four benchmarks.
vision-language models multi-modal understanding generative images synthetic images prompt inversion style retrieval image captioning visual question answering JourneyDB