Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections
  of Images
We present a system using Multimodal LLMs (MLLMs) to analyze a large database
with tens of millions of images captured at different times, with the aim of
discovering patterns in temporal changes. Specifically, we aim to capture
frequent co-occurring changes ("trends") across a city over a certain period.
Unlike previous visual analyses, our analysis answers open-ended queries (e.g.,
"what are the frequent types of changes in the city?") without any
predetermined target subjects or training labels. These properties cast prior
learning-based or unsupervised visual analysis tools unsuitable. We identify
MLLMs as a novel tool for their open-ended semantic understanding capabilities.
Yet, our datasets are four orders of magnitude too large for an MLLM to ingest
as context. So we introduce a bottom-up procedure that decomposes the massive
visual analysis problem into more tractable sub-problems. We carefully design
MLLM-based solutions to each sub-problem. During experiments and ablation
studies with our system, we find it significantly outperforms baselines and is
able to discover interesting trends from images captured in large cities (e.g.,
"addition of outdoor dining,", "overpass was painted blue," etc.). See more
results and interactive demos at https://boyangdeng.com/visual-chronicles.
67fd083b3bf3cc207a879c3d
Boyang Deng
claimed_verified
2025-04-14T13:17:32.224Z
66d3896c6839bd191d0c5ad3
/avatars/34acf042c946b5e7144ce3c002c9939f.svg
Boyang Deng
user
bydeng
67fd083b3bf3cc207a879c3e
Songyou Peng
claimed_verified
2025-04-14T18:40:03.787Z
620cb0722d8bc91e1cf2e6e7
https://cdn-avatars.huggingface.co/v1/production/uploads/1644998719617-noauth.jpeg
Songyou Peng
user
syp
67fd083b3bf3cc207a879c3f
Kyle Genova
67fd083b3bf3cc207a879c40
Gordon Wetzstein
67fd083b3bf3cc207a879c41
Noah Snavely
67fd083b3bf3cc207a879c42
Leonidas Guibas
67fd083b3bf3cc207a879c43
Thomas Funkhouser
Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections
  of Images
We present a system using Multimodal LLMs (MLLMs) to analyze a large database
with tens of millions of images captured at different times, with the aim of
discovering patterns in temporal changes. Specifically, we aim to capture
frequent co-occurring changes ("trends") across a city over a certain period.
Unlike previous visual analyses, our analysis answers open-ended queries (e.g.,
"what are the frequent types of changes in the city?") without any
predetermined target subjects or training labels. These properties cast prior
learning-based or unsupervised visual analysis tools unsuitable. We identify
MLLMs as a novel tool for their open-ended semantic understanding capabilities.
Yet, our datasets are four orders of magnitude too large for an MLLM to ingest
as context. So we introduce a bottom-up procedure that decomposes the massive
visual analysis problem into more tractable sub-problems. We carefully design
MLLM-based solutions to each sub-problem. During experiments and ablation
studies with our system, we find it significantly outperforms baselines and is
able to discover interesting trends from images captured in large cities (e.g.,
"addition of outdoor dining,", "overpass was painted blue," etc.). See more
results and interactive demos at https://boyangdeng.com/visual-chronicles.
A system using Multimodal LLMs analyzes large image datasets to discover open-ended temporal patterns and trends without predefined labels, achieving superior performance compared to baselines.
Multimodal LLMs MLLMs visual analysis open-ended queries temporal patterns trends bottom-up procedure ablation studies