title: SEAGULL: No-reference Image Quality Assessment for Regions of Interest
    via Vision-Language Instruction Tuning
summary: Existing Image Quality Assessment (IQA) methods achieve remarkable success in
  analyzing quality for overall image, but few works explore quality analysis for
  Regions of Interest (ROIs). The quality analysis of ROIs can provide
  fine-grained guidance for image quality improvement and is crucial for
  scenarios focusing on region-level quality. This paper proposes a novel
  network, SEAGULL, which can SEe and Assess ROIs quality with GUidance from a
  Large vision-Language model. SEAGULL incorporates a vision-language model
  (VLM), masks generated by Segment Anything Model (SAM) to specify ROIs, and a
  meticulously designed Mask-based Feature Extractor (MFE) to extract global and
  local tokens for specified ROIs, enabling accurate fine-grained IQA for ROIs.
  Moreover, this paper constructs two ROI-based IQA datasets, SEAGULL-100w and
  SEAGULL-3k, for training and evaluating ROI-based IQA. SEAGULL-100w comprises
  about 100w synthetic distortion images with 33 million ROIs for pre-training to
  improve the model's ability of regional quality perception, and SEAGULL-3k
  contains about 3k authentic distortion ROIs to enhance the model's ability to
  perceive real world distortions. After pre-training on SEAGULL-100w and
  fine-tuning on SEAGULL-3k, SEAGULL shows remarkable performance on fine-grained
  ROI quality assessment. Code and datasets are publicly available at the
  https://github.com/chencn2020/Seagull.
paper_authors._id: 673d449ca2f8aff91ef2615a
paper_authors.name: Zewen Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-20T10:08:51.252Z
paper_authors._id: 63e78d4edb40d9e67fedad16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63e78d4edb40d9e67fedad16/7QjxNI7p7iZD_6s0TZvTD.jpeg
paper_authors.fullname: Zevin
paper_authors.type: user
paper_authors.user: Zevin2023
paper_authors._id: 673d449ca2f8aff91ef2615b
paper_authors.name: Juan Wang
paper_authors._id: 673d449ca2f8aff91ef2615c
paper_authors.name: Wen Wang
paper_authors._id: 673d449ca2f8aff91ef2615d
paper_authors.name: Sunhan Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-20T13:42:43.952Z
paper_authors._id: 661d280abca1a6038b2bfe5e
paper_authors.avatarUrl: /avatars/62b4818824c6c0ef9afad0d727618681.svg
paper_authors.fullname: Sunhan Xu
paper_authors.type: user
paper_authors.user: Shuanghan
paper_authors._id: 673d449ca2f8aff91ef2615e
paper_authors.name: Hang Xiong
paper_authors._id: 673d449ca2f8aff91ef2615f
paper_authors.name: Yun Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-20T13:54:47.572Z
paper_authors._id: 66a0bc420b187dd45f7bf08f
paper_authors.avatarUrl: /avatars/31bc135113862d932958607fdccd9c4b.svg
paper_authors.fullname: Yun Zeng
paper_authors.type: user
paper_authors.user: Yun0418
paper_authors._id: 673d449ca2f8aff91ef26160
paper_authors.name: Jian Guo
paper_authors._id: 673d449ca2f8aff91ef26161
paper_authors.name: Shuxun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-20T13:55:23.017Z
paper_authors._id: 6459f8cac70a926dc46b10d1
paper_authors.avatarUrl: /avatars/d68619fb59edd7a93a9709695b12c94d.svg
paper_authors.fullname: wangshuxun
paper_authors.type: user
paper_authors.user: wangshuxun
paper_authors._id: 673d449ca2f8aff91ef26162
paper_authors.name: Chunfeng Yuan
paper_authors._id: 673d449ca2f8aff91ef26163
paper_authors.name: Bing Li
paper_authors._id: 673d449ca2f8aff91ef26164
paper_authors.name: Weiming Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-20T13:56:57.438Z
paper_authors._id: 6641c7c05aa8e7031cc6afc0
paper_authors.avatarUrl: /avatars/dcf2b101a7afab828bc308f922648e3f.svg
paper_authors.fullname: Weiming Hu
paper_authors.type: user
paper_authors.user: huweim
paper_title: SEAGULL: No-reference Image Quality Assessment for Regions of Interest
    via Vision-Language Instruction Tuning
paper_summary: Existing Image Quality Assessment (IQA) methods achieve remarkable success in
  analyzing quality for overall image, but few works explore quality analysis for
  Regions of Interest (ROIs). The quality analysis of ROIs can provide
  fine-grained guidance for image quality improvement and is crucial for
  scenarios focusing on region-level quality. This paper proposes a novel
  network, SEAGULL, which can SEe and Assess ROIs quality with GUidance from a
  Large vision-Language model. SEAGULL incorporates a vision-language model
  (VLM), masks generated by Segment Anything Model (SAM) to specify ROIs, and a
  meticulously designed Mask-based Feature Extractor (MFE) to extract global and
  local tokens for specified ROIs, enabling accurate fine-grained IQA for ROIs.
  Moreover, this paper constructs two ROI-based IQA datasets, SEAGULL-100w and
  SEAGULL-3k, for training and evaluating ROI-based IQA. SEAGULL-100w comprises
  about 100w synthetic distortion images with 33 million ROIs for pre-training to
  improve the model's ability of regional quality perception, and SEAGULL-3k
  contains about 3k authentic distortion ROIs to enhance the model's ability to
  perceive real world distortions. After pre-training on SEAGULL-100w and
  fine-tuning on SEAGULL-3k, SEAGULL shows remarkable performance on fine-grained
  ROI quality assessment. Code and datasets are publicly available at the
  https://github.com/chencn2020/Seagull.
paper_ai_keywords: vision-language model Segment Anything Model (SAM) Mask-based Feature Extractor (MFE) global tokens local tokens fine-grained IQA ROI-based IQA SEAGULL-100w SEAGULL-3k pre-training fine-tuning