title: RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image
    Generation
summary: Subject-driven text-to-image (T2I) generation aims to produce images that
  align with a given textual description, while preserving the visual identity
  from a referenced subject image. Despite its broad downstream applicability --
  ranging from enhanced personalization in image generation to consistent
  character representation in video rendering -- progress in this field is
  limited by the lack of reliable automatic evaluation. Existing methods either
  assess only one aspect of the task (i.e., textual alignment or subject
  preservation), misalign with human judgments, or rely on costly API-based
  evaluation. To address this, we introduce RefVNLI, a cost-effective metric that
  evaluates both textual alignment and subject preservation in a single
  prediction. Trained on a large-scale dataset derived from video-reasoning
  benchmarks and image perturbations, RefVNLI outperforms or matches existing
  baselines across multiple benchmarks and subject categories (e.g.,
  Animal, Object), achieving up to 6.4-point gains in textual
  alignment and 8.5-point gains in subject consistency. It also excels with
  lesser-known concepts, aligning with human preferences at over 87\% accuracy.
paper_authors._id: 680b44fb426b7d5bc2018c75
paper_authors.name: Aviv Slobodkin
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-06-07T08:09:16.083Z
paper_authors._id: 631da07f6d6a5870f3d2c375
paper_authors.avatarUrl: /avatars/242e344dca08057bdf1eef09f69b41b2.svg
paper_authors.fullname: Aviv Slobodkin
paper_authors.type: user
paper_authors.user: lovodkin93
paper_authors._id: 680b44fb426b7d5bc2018c76
paper_authors.name: Hagai Taitelbaum
paper_authors._id: 680b44fb426b7d5bc2018c77
paper_authors.name: Yonatan Bitton
paper_authors._id: 680b44fb426b7d5bc2018c78
paper_authors.name: Brian Gordon
paper_authors._id: 680b44fb426b7d5bc2018c79
paper_authors.name: Michal Sokolik
paper_authors._id: 680b44fb426b7d5bc2018c7a
paper_authors.name: Nitzan Bitton Guetta
paper_authors._id: 680b44fb426b7d5bc2018c7b
paper_authors.name: Almog Gueta
paper_authors._id: 680b44fb426b7d5bc2018c7c
paper_authors.name: Royi Rassin
paper_authors._id: 680b44fb426b7d5bc2018c7d
paper_authors.name: Itay Laish
paper_authors._id: 680b44fb426b7d5bc2018c7e
paper_authors.name: Dani Lischinski
paper_authors._id: 680b44fb426b7d5bc2018c7f
paper_authors.name: Idan Szpektor
paper_title: RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image
    Generation
paper_summary: Subject-driven text-to-image (T2I) generation aims to produce images that
  align with a given textual description, while preserving the visual identity
  from a referenced subject image. Despite its broad downstream applicability --
  ranging from enhanced personalization in image generation to consistent
  character representation in video rendering -- progress in this field is
  limited by the lack of reliable automatic evaluation. Existing methods either
  assess only one aspect of the task (i.e., textual alignment or subject
  preservation), misalign with human judgments, or rely on costly API-based
  evaluation. To address this, we introduce RefVNLI, a cost-effective metric that
  evaluates both textual alignment and subject preservation in a single
  prediction. Trained on a large-scale dataset derived from video-reasoning
  benchmarks and image perturbations, RefVNLI outperforms or matches existing
  baselines across multiple benchmarks and subject categories (e.g.,
  Animal, Object), achieving up to 6.4-point gains in textual
  alignment and 8.5-point gains in subject consistency. It also excels with
  lesser-known concepts, aligning with human preferences at over 87\% accuracy.
paper_ai_summary: RefVNLI, a novel metric, evaluates both textual alignment and subject preservation in subject-driven text-to-image generation, outperforming existing methods across various benchmarks and subject categories.
paper_ai_keywords: text-to-image T2I textual alignment subject preservation RefVNLI video-reasoning benchmarks image perturbations