title: Lost in Time: Clock and Calendar Understanding Challenges in Multimodal
    LLMs
summary: Understanding time from visual representations is a fundamental cognitive
  skill, yet it remains a challenge for multimodal large language models (MLLMs).
  In this work, we investigate the capabilities of MLLMs in interpreting time and
  date through analogue clocks and yearly calendars. To facilitate this, we
  curated a structured dataset comprising two subsets: 1) ClockQA,
  which comprises various types of clock styles-standard, black-dial,
  no-second-hand, Roman numeral, and arrow-hand clocks-paired with time related
  questions; and 2) CalendarQA, which consists of yearly calendar
  images with questions ranging from commonly known dates (e.g., Christmas, New
  Year's Day) to computationally derived ones (e.g., the 100th or 153rd day of
  the year). We aim to analyse how MLLMs can perform visual recognition,
  numerical reasoning, and temporal inference when presented with time-related
  visual data. Our evaluations show that despite recent advancements, reliably
  understanding time remains a significant challenge for MLLMs.
paper_authors._id: 67aa05c5ffb9f6b5b2f658b2
paper_authors.name: Rohit Saxena
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:18:07.805Z
paper_authors._id: 657ccbf2869d5bb0e53b482f
paper_authors.avatarUrl: /avatars/2eae5a10bdc14814a04d9f255f16de6b.svg
paper_authors.fullname: Rohit Saxena
paper_authors.type: user
paper_authors.user: rohitsaxena
paper_authors._id: 67aa05c5ffb9f6b5b2f658b3
paper_authors.name: Aryo Pradipta Gema
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-13T08:26:03.971Z
paper_authors._id: 644f895e23d7eb05ca695054
paper_authors.avatarUrl: /avatars/3fb04dd8544b403262bf98507de05453.svg
paper_authors.fullname: Aryo Pradipta Gema
paper_authors.type: user
paper_authors.user: aryopg
paper_authors._id: 67aa05c5ffb9f6b5b2f658b4
paper_authors.name: Pasquale Minervini
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-10T15:51:20.711Z
paper_authors._id: 61001311e043e15c13412d30
paper_authors.avatarUrl: /avatars/eea1e4c39decee282f2940d122090491.svg
paper_authors.fullname: Pasquale Minervini
paper_authors.type: user
paper_authors.user: pminervini
paper_title: Lost in Time: Clock and Calendar Understanding Challenges in Multimodal
    LLMs
paper_summary: Understanding time from visual representations is a fundamental cognitive
  skill, yet it remains a challenge for multimodal large language models (MLLMs).
  In this work, we investigate the capabilities of MLLMs in interpreting time and
  date through analogue clocks and yearly calendars. To facilitate this, we
  curated a structured dataset comprising two subsets: 1) ClockQA,
  which comprises various types of clock styles-standard, black-dial,
  no-second-hand, Roman numeral, and arrow-hand clocks-paired with time related
  questions; and 2) CalendarQA, which consists of yearly calendar
  images with questions ranging from commonly known dates (e.g., Christmas, New
  Year's Day) to computationally derived ones (e.g., the 100th or 153rd day of
  the year). We aim to analyse how MLLMs can perform visual recognition,
  numerical reasoning, and temporal inference when presented with time-related
  visual data. Our evaluations show that despite recent advancements, reliably
  understanding time remains a significant challenge for MLLMs.
paper_ai_keywords: diffusion models multimodal large language models (MLLMs) ClockQA CalendarQA visual recognition numerical reasoning temporal inference