title: Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections
    of Images
summary: We present a system using Multimodal LLMs (MLLMs) to analyze a large database
  with tens of millions of images captured at different times, with the aim of
  discovering patterns in temporal changes. Specifically, we aim to capture
  frequent co-occurring changes ("trends") across a city over a certain period.
  Unlike previous visual analyses, our analysis answers open-ended queries (e.g.,
  "what are the frequent types of changes in the city?") without any
  predetermined target subjects or training labels. These properties cast prior
  learning-based or unsupervised visual analysis tools unsuitable. We identify
  MLLMs as a novel tool for their open-ended semantic understanding capabilities.
  Yet, our datasets are four orders of magnitude too large for an MLLM to ingest
  as context. So we introduce a bottom-up procedure that decomposes the massive
  visual analysis problem into more tractable sub-problems. We carefully design
  MLLM-based solutions to each sub-problem. During experiments and ablation
  studies with our system, we find it significantly outperforms baselines and is
  able to discover interesting trends from images captured in large cities (e.g.,
  "addition of outdoor dining,", "overpass was painted blue," etc.). See more
  results and interactive demos at https://boyangdeng.com/visual-chronicles.
paper_authors._id: 67fd083b3bf3cc207a879c3d
paper_authors.name: Boyang Deng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-14T13:17:32.224Z
paper_authors._id: 66d3896c6839bd191d0c5ad3
paper_authors.avatarUrl: /avatars/34acf042c946b5e7144ce3c002c9939f.svg
paper_authors.fullname: Boyang Deng
paper_authors.type: user
paper_authors.user: bydeng
paper_authors._id: 67fd083b3bf3cc207a879c3e
paper_authors.name: Songyou Peng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-14T18:40:03.787Z
paper_authors._id: 620cb0722d8bc91e1cf2e6e7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644998719617-noauth.jpeg
paper_authors.fullname: Songyou Peng
paper_authors.type: user
paper_authors.user: syp
paper_authors._id: 67fd083b3bf3cc207a879c3f
paper_authors.name: Kyle Genova
paper_authors._id: 67fd083b3bf3cc207a879c40
paper_authors.name: Gordon Wetzstein
paper_authors._id: 67fd083b3bf3cc207a879c41
paper_authors.name: Noah Snavely
paper_authors._id: 67fd083b3bf3cc207a879c42
paper_authors.name: Leonidas Guibas
paper_authors._id: 67fd083b3bf3cc207a879c43
paper_authors.name: Thomas Funkhouser
paper_title: Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections
    of Images
paper_summary: We present a system using Multimodal LLMs (MLLMs) to analyze a large database
  with tens of millions of images captured at different times, with the aim of
  discovering patterns in temporal changes. Specifically, we aim to capture
  frequent co-occurring changes ("trends") across a city over a certain period.
  Unlike previous visual analyses, our analysis answers open-ended queries (e.g.,
  "what are the frequent types of changes in the city?") without any
  predetermined target subjects or training labels. These properties cast prior
  learning-based or unsupervised visual analysis tools unsuitable. We identify
  MLLMs as a novel tool for their open-ended semantic understanding capabilities.
  Yet, our datasets are four orders of magnitude too large for an MLLM to ingest
  as context. So we introduce a bottom-up procedure that decomposes the massive
  visual analysis problem into more tractable sub-problems. We carefully design
  MLLM-based solutions to each sub-problem. During experiments and ablation
  studies with our system, we find it significantly outperforms baselines and is
  able to discover interesting trends from images captured in large cities (e.g.,
  "addition of outdoor dining,", "overpass was painted blue," etc.). See more
  results and interactive demos at https://boyangdeng.com/visual-chronicles.
paper_ai_summary: A system using Multimodal LLMs analyzes large image datasets to discover open-ended temporal patterns and trends without predefined labels, achieving superior performance compared to baselines.
paper_ai_keywords: Multimodal LLMs MLLMs visual analysis open-ended queries temporal patterns trends bottom-up procedure ablation studies