title: VideoGameQA-Bench: Evaluating Vision-Language Models for Video Game
    Quality Assurance
summary: With video games now generating the highest revenues in the entertainment
  industry, optimizing game development workflows has become essential for the
  sector's sustained growth. Recent advancements in Vision-Language Models (VLMs)
  offer considerable potential to automate and enhance various aspects of game
  development, particularly Quality Assurance (QA), which remains one of the
  industry's most labor-intensive processes with limited automation options. To
  accurately evaluate the performance of VLMs in video game QA tasks and
  determine their effectiveness in handling real-world scenarios, there is a
  clear need for standardized benchmarks, as existing benchmarks are insufficient
  to address the specific requirements of this domain. To bridge this gap, we
  introduce VideoGameQA-Bench, a comprehensive benchmark that covers a wide array
  of game QA activities, including visual unit testing, visual regression
  testing, needle-in-a-haystack tasks, glitch detection, and bug report
  generation for both images and videos of various games. Code and data are
  available at: https://asgaardlab.github.io/videogameqa-bench/
paper_authors._id: 682fe833f39f561d1d8cd5d1
paper_authors.name: Mohammad Reza Taesiri
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-23T03:18:36.072Z
paper_authors._id: 6039478ab3ecf716b1a5fd4d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg
paper_authors.fullname: taesiri
paper_authors.type: user
paper_authors.user: taesiri
paper_authors._id: 682fe833f39f561d1d8cd5d2
paper_authors.name: Abhijay Ghildyal
paper_authors._id: 682fe833f39f561d1d8cd5d3
paper_authors.name: Saman Zadtootaghaj
paper_authors._id: 682fe833f39f561d1d8cd5d4
paper_authors.name: Nabajeet Barman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:15:17.798Z
paper_authors._id: 66605d1588f33df0d836853a
paper_authors.avatarUrl: /avatars/ea73706c2410c03b8ba5dd89d929f483.svg
paper_authors.fullname: Nabajeet Barman
paper_authors.type: user
paper_authors.user: nbarman
paper_authors._id: 682fe833f39f561d1d8cd5d5
paper_authors.name: Cor-Paul Bezemer
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-23T14:19:43.747Z
paper_authors._id: 644feede17b6189cda58575d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/WVo1Ah7xmHEeBOUQpkYgS.png
paper_authors.fullname: Cor-Paul
paper_authors.type: user
paper_authors.user: corpaul
paper_title: VideoGameQA-Bench: Evaluating Vision-Language Models for Video Game
    Quality Assurance
paper_summary: With video games now generating the highest revenues in the entertainment
  industry, optimizing game development workflows has become essential for the
  sector's sustained growth. Recent advancements in Vision-Language Models (VLMs)
  offer considerable potential to automate and enhance various aspects of game
  development, particularly Quality Assurance (QA), which remains one of the
  industry's most labor-intensive processes with limited automation options. To
  accurately evaluate the performance of VLMs in video game QA tasks and
  determine their effectiveness in handling real-world scenarios, there is a
  clear need for standardized benchmarks, as existing benchmarks are insufficient
  to address the specific requirements of this domain. To bridge this gap, we
  introduce VideoGameQA-Bench, a comprehensive benchmark that covers a wide array
  of game QA activities, including visual unit testing, visual regression
  testing, needle-in-a-haystack tasks, glitch detection, and bug report
  generation for both images and videos of various games. Code and data are
  available at: https://asgaardlab.github.io/videogameqa-bench/
paper_ai_summary: A benchmark called VideoGameQA-Bench is introduced to assess Vision-Language Models in video game quality assurance tasks.
paper_ai_keywords: Vision-Language Models VideoGameQA-Bench visual unit testing visual regression testing needle-in-a-haystack tasks glitch detection bug report generation