title: Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Names
summary: Enabling engagement of manga by visually impaired individuals presents a
  significant challenge due to its inherently visual nature. With the goal of
  fostering accessibility, this paper aims to generate a dialogue transcript of a
  complete manga chapter, entirely automatically, with a particular emphasis on
  ensuring narrative consistency. This entails identifying (i) what is being
  said, i.e., detecting the texts on each page and classifying them into
  essential vs non-essential, and (ii) who is saying it, i.e., attributing each
  dialogue to its speaker, while ensuring the same characters are named
  consistently throughout the chapter.
    To this end, we introduce: (i) Magiv2, a model that is capable of generating
  high-quality chapter-wide manga transcripts with named characters and
  significantly higher precision in speaker diarisation over prior works; (ii) an
  extension of the PopManga evaluation dataset, which now includes annotations
  for speech-bubble tail boxes, associations of text to corresponding tails,
  classifications of text as essential or non-essential, and the identity for
  each character box; and (iii) a new character bank dataset, which comprises
  over 11K characters from 76 manga series, featuring 11.5K exemplar character
  images in total, as well as a list of chapters in which they appear. The code,
  trained model, and both datasets can be found at:
  https://github.com/ragavsachdeva/magi
paper_authors._id: 66ac3dbd4b7cf96c1c8f43ae
paper_authors.name: Ragav Sachdeva
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:12:59.043Z
paper_authors._id: 630852d2f0dc38fb47c347a4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630852d2f0dc38fb47c347a4/NqkR5EUT45x-Ucx79XOXY.png
paper_authors.fullname: Ragav Sachdeva
paper_authors.type: user
paper_authors.user: ragavsachdeva
paper_authors._id: 66ac3dbd4b7cf96c1c8f43af
paper_authors.name: Gyungin Shin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:13:04.673Z
paper_authors._id: 6238c507254d95e12bd1952b
paper_authors.avatarUrl: /avatars/8cea1c761c7b2e996277727143bfbea2.svg
paper_authors.fullname: Gyungin Shin
paper_authors.type: user
paper_authors.user: noelshin
paper_authors._id: 66ac3dbd4b7cf96c1c8f43b0
paper_authors.name: Andrew Zisserman
paper_title: Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Names
paper_summary: Enabling engagement of manga by visually impaired individuals presents a
  significant challenge due to its inherently visual nature. With the goal of
  fostering accessibility, this paper aims to generate a dialogue transcript of a
  complete manga chapter, entirely automatically, with a particular emphasis on
  ensuring narrative consistency. This entails identifying (i) what is being
  said, i.e., detecting the texts on each page and classifying them into
  essential vs non-essential, and (ii) who is saying it, i.e., attributing each
  dialogue to its speaker, while ensuring the same characters are named
  consistently throughout the chapter.
    To this end, we introduce: (i) Magiv2, a model that is capable of generating
  high-quality chapter-wide manga transcripts with named characters and
  significantly higher precision in speaker diarisation over prior works; (ii) an
  extension of the PopManga evaluation dataset, which now includes annotations
  for speech-bubble tail boxes, associations of text to corresponding tails,
  classifications of text as essential or non-essential, and the identity for
  each character box; and (iii) a new character bank dataset, which comprises
  over 11K characters from 76 manga series, featuring 11.5K exemplar character
  images in total, as well as a list of chapters in which they appear. The code,
  trained model, and both datasets can be found at:
  https://github.com/ragavsachdeva/magi
paper_ai_summary: A model named Magiv2 is introduced to automatically generate dialogue transcripts for manga, ensuring narrative consistency and high precision in speaker identification.
paper_ai_keywords: Magiv2 speaker diarisation PopManga dataset character bank dataset