VideoGameQA-Bench: Evaluating Vision-Language Models for Video Game
  Quality Assurance
With video games now generating the highest revenues in the entertainment
industry, optimizing game development workflows has become essential for the
sector's sustained growth. Recent advancements in Vision-Language Models (VLMs)
offer considerable potential to automate and enhance various aspects of game
development, particularly Quality Assurance (QA), which remains one of the
industry's most labor-intensive processes with limited automation options. To
accurately evaluate the performance of VLMs in video game QA tasks and
determine their effectiveness in handling real-world scenarios, there is a
clear need for standardized benchmarks, as existing benchmarks are insufficient
to address the specific requirements of this domain. To bridge this gap, we
introduce VideoGameQA-Bench, a comprehensive benchmark that covers a wide array
of game QA activities, including visual unit testing, visual regression
testing, needle-in-a-haystack tasks, glitch detection, and bug report
generation for both images and videos of various games. Code and data are
available at: https://asgaardlab.github.io/videogameqa-bench/
682fe833f39f561d1d8cd5d1
Mohammad Reza Taesiri
extracted_confirmed
2025-05-23T03:18:36.072Z
6039478ab3ecf716b1a5fd4d
https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg
taesiri
user
taesiri
682fe833f39f561d1d8cd5d2
Abhijay Ghildyal
682fe833f39f561d1d8cd5d3
Saman Zadtootaghaj
682fe833f39f561d1d8cd5d4
Nabajeet Barman
claimed_verified
2025-05-26T08:15:17.798Z
66605d1588f33df0d836853a
/avatars/ea73706c2410c03b8ba5dd89d929f483.svg
Nabajeet Barman
user
nbarman
682fe833f39f561d1d8cd5d5
Cor-Paul Bezemer
extracted_confirmed
2025-05-23T14:19:43.747Z
644feede17b6189cda58575d
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/WVo1Ah7xmHEeBOUQpkYgS.png
Cor-Paul
user
corpaul
VideoGameQA-Bench: Evaluating Vision-Language Models for Video Game
  Quality Assurance
With video games now generating the highest revenues in the entertainment
industry, optimizing game development workflows has become essential for the
sector's sustained growth. Recent advancements in Vision-Language Models (VLMs)
offer considerable potential to automate and enhance various aspects of game
development, particularly Quality Assurance (QA), which remains one of the
industry's most labor-intensive processes with limited automation options. To
accurately evaluate the performance of VLMs in video game QA tasks and
determine their effectiveness in handling real-world scenarios, there is a
clear need for standardized benchmarks, as existing benchmarks are insufficient
to address the specific requirements of this domain. To bridge this gap, we
introduce VideoGameQA-Bench, a comprehensive benchmark that covers a wide array
of game QA activities, including visual unit testing, visual regression
testing, needle-in-a-haystack tasks, glitch detection, and bug report
generation for both images and videos of various games. Code and data are
available at: https://asgaardlab.github.io/videogameqa-bench/
A benchmark called VideoGameQA-Bench is introduced to assess Vision-Language Models in video game quality assurance tasks.
Vision-Language Models VideoGameQA-Bench visual unit testing visual regression testing needle-in-a-haystack tasks glitch detection bug report generation