title: DateLogicQA: Benchmarking Temporal Biases in Large Language Models
summary: This paper introduces DateLogicQA, a benchmark with 190 questions covering
  diverse date formats, temporal contexts, and reasoning types. We propose the
  Semantic Integrity Metric to assess tokenization quality and analyse two
  biases: Representation-Level Bias, affecting embeddings, and Logical-Level
  Bias, influencing reasoning outputs. Our findings provide a comprehensive
  evaluation of LLMs' capabilities and limitations in temporal reasoning,
  highlighting key challenges in handling temporal data accurately. The GitHub
  repository for our work is available at
  https://github.com/gagan3012/EAIS-Temporal-Bias
paper_authors._id: 676577a8abcd70b404ad67ca
paper_authors.name: Gagan Bhatia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T15:42:23.951Z
paper_authors._id: 60394599033b61166496163b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1614366097007-noauth.jpeg
paper_authors.fullname: Gagan Bhatia
paper_authors.type: user
paper_authors.user: gagan3012
paper_authors._id: 676577a8abcd70b404ad67cb
paper_authors.name: MingZe Tang
paper_authors._id: 676577a8abcd70b404ad67cc
paper_authors.name: Cristina Mahanta
paper_authors._id: 676577a8abcd70b404ad67cd
paper_authors.name: Madiha Kazi
paper_title: DateLogicQA: Benchmarking Temporal Biases in Large Language Models
paper_summary: This paper introduces DateLogicQA, a benchmark with 190 questions covering
  diverse date formats, temporal contexts, and reasoning types. We propose the
  Semantic Integrity Metric to assess tokenization quality and analyse two
  biases: Representation-Level Bias, affecting embeddings, and Logical-Level
  Bias, influencing reasoning outputs. Our findings provide a comprehensive
  evaluation of LLMs' capabilities and limitations in temporal reasoning,
  highlighting key challenges in handling temporal data accurately. The GitHub
  repository for our work is available at
  https://github.com/gagan3012/EAIS-Temporal-Bias
paper_ai_keywords: Semantic Integrity Metric Representation-Level Bias Logical-Level Bias LLMs (Large Language Models) temporal reasoning