title: One missing piece in Vision and Language: A Survey on Comics
    Understanding
summary: Vision-language models have recently evolved into versatile systems capable
  of high performance across a range of tasks, such as document understanding,
  visual question answering, and grounding, often in zero-shot settings. Comics
  Understanding, a complex and multifaceted field, stands to greatly benefit from
  these advances. Comics, as a medium, combine rich visual and textual
  narratives, challenging AI models with tasks that span image classification,
  object detection, instance segmentation, and deeper narrative comprehension
  through sequential panels. However, the unique structure of comics --
  characterized by creative variations in style, reading order, and non-linear
  storytelling -- presents a set of challenges distinct from those in other
  visual-language domains. In this survey, we present a comprehensive review of
  Comics Understanding from both dataset and task perspectives. Our contributions
  are fivefold: (1) We analyze the structure of the comics medium, detailing its
  distinctive compositional elements; (2) We survey the widely used datasets and
  tasks in comics research, emphasizing their role in advancing the field; (3) We
  introduce the Layer of Comics Understanding (LoCU) framework, a novel taxonomy
  that redefines vision-language tasks within comics and lays the foundation for
  future work; (4) We provide a detailed review and categorization of existing
  methods following the LoCU framework; (5) Finally, we highlight current
  research challenges and propose directions for future exploration, particularly
  in the context of vision-language models applied to comics. This survey is the
  first to propose a task-oriented framework for comics intelligence and aims to
  guide future research by addressing critical gaps in data availability and task
  definition. A project associated with this survey is available at
  https://github.com/emanuelevivoli/awesome-comics-understanding.
paper_authors._id: 66e90e49955eeec85e4f6842
paper_authors.name: Emanuele Vivoli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-17T09:30:07.682Z
paper_authors._id: 6065a9cbe43e52694178ed78
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6065a9cbe43e52694178ed78/TwnrOfTacrwtLnBC8cS8e.jpeg
paper_authors.fullname: Emanuele Vivoli
paper_authors.type: user
paper_authors.user: emanuelevivoli
paper_authors._id: 66e90e49955eeec85e4f6843
paper_authors.name: Andrey Barsky
paper_authors._id: 66e90e49955eeec85e4f6844
paper_authors.name: Mohamed Ali Souibgui
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-17T15:08:12.305Z
paper_authors._id: 66e98dcf899bdb384bd951d6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66e98dcf899bdb384bd951d6/6nCMkogUuQWJm3FR05Ygc.jpeg
paper_authors.fullname: Mohamed Ali Souibgui
paper_authors.type: user
paper_authors.user: msouibgui
paper_authors._id: 66e90e49955eeec85e4f6845
paper_authors.name: Artemis LLabres
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-17T11:31:49.342Z
paper_authors._id: 630f3e904dbbbb6f667eceb7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630f3e904dbbbb6f667eceb7/GENlN7kDSoGhmIL7N3C8h.png
paper_authors.fullname: Artemis Llabrés
paper_authors.type: user
paper_authors.user: Llabres
paper_authors._id: 66e90e49955eeec85e4f6846
paper_authors.name: Marco Bertini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-17T09:31:38.153Z
paper_authors._id: 6319e9b78b8425c8fe653dd6
paper_authors.avatarUrl: /avatars/b1abd8bc31e7e39d7d563cc275656e6c.svg
paper_authors.fullname: Marco Bertini
paper_authors.type: user
paper_authors.user: MarcoBertini
paper_authors._id: 66e90e49955eeec85e4f6847
paper_authors.name: Dimosthenis Karatzas
paper_title: One missing piece in Vision and Language: A Survey on Comics
    Understanding
paper_summary: Vision-language models have recently evolved into versatile systems capable
  of high performance across a range of tasks, such as document understanding,
  visual question answering, and grounding, often in zero-shot settings. Comics
  Understanding, a complex and multifaceted field, stands to greatly benefit from
  these advances. Comics, as a medium, combine rich visual and textual
  narratives, challenging AI models with tasks that span image classification,
  object detection, instance segmentation, and deeper narrative comprehension
  through sequential panels. However, the unique structure of comics --
  characterized by creative variations in style, reading order, and non-linear
  storytelling -- presents a set of challenges distinct from those in other
  visual-language domains. In this survey, we present a comprehensive review of
  Comics Understanding from both dataset and task perspectives. Our contributions
  are fivefold: (1) We analyze the structure of the comics medium, detailing its
  distinctive compositional elements; (2) We survey the widely used datasets and
  tasks in comics research, emphasizing their role in advancing the field; (3) We
  introduce the Layer of Comics Understanding (LoCU) framework, a novel taxonomy
  that redefines vision-language tasks within comics and lays the foundation for
  future work; (4) We provide a detailed review and categorization of existing
  methods following the LoCU framework; (5) Finally, we highlight current
  research challenges and propose directions for future exploration, particularly
  in the context of vision-language models applied to comics. This survey is the
  first to propose a task-oriented framework for comics intelligence and aims to
  guide future research by addressing critical gaps in data availability and task
  definition. A project associated with this survey is available at
  https://github.com/emanuelevivoli/awesome-comics-understanding.
paper_ai_summary: A comprehensive survey addresses the challenges of comics understanding, introducing a new taxonomy and reviewing methods within the context of vision-language models.
paper_ai_keywords: vision-language models document understanding visual question answering grounding Comics Understanding image classification object detection instance segmentation narrative comprehension sequential panels Layer of Comics Understanding (LoCU) framework