SEAGULL: No-reference Image Quality Assessment for Regions of Interest
  via Vision-Language Instruction Tuning
Existing Image Quality Assessment (IQA) methods achieve remarkable success in
analyzing quality for overall image, but few works explore quality analysis for
Regions of Interest (ROIs). The quality analysis of ROIs can provide
fine-grained guidance for image quality improvement and is crucial for
scenarios focusing on region-level quality. This paper proposes a novel
network, SEAGULL, which can SEe and Assess ROIs quality with GUidance from a
Large vision-Language model. SEAGULL incorporates a vision-language model
(VLM), masks generated by Segment Anything Model (SAM) to specify ROIs, and a
meticulously designed Mask-based Feature Extractor (MFE) to extract global and
local tokens for specified ROIs, enabling accurate fine-grained IQA for ROIs.
Moreover, this paper constructs two ROI-based IQA datasets, SEAGULL-100w and
SEAGULL-3k, for training and evaluating ROI-based IQA. SEAGULL-100w comprises
about 100w synthetic distortion images with 33 million ROIs for pre-training to
improve the model's ability of regional quality perception, and SEAGULL-3k
contains about 3k authentic distortion ROIs to enhance the model's ability to
perceive real world distortions. After pre-training on SEAGULL-100w and
fine-tuning on SEAGULL-3k, SEAGULL shows remarkable performance on fine-grained
ROI quality assessment. Code and datasets are publicly available at the
https://github.com/chencn2020/Seagull.
673d449ca2f8aff91ef2615a
Zewen Chen
claimed_verified
2024-11-20T10:08:51.252Z
63e78d4edb40d9e67fedad16
https://cdn-avatars.huggingface.co/v1/production/uploads/63e78d4edb40d9e67fedad16/7QjxNI7p7iZD_6s0TZvTD.jpeg
Zevin
user
Zevin2023
673d449ca2f8aff91ef2615b
Juan Wang
673d449ca2f8aff91ef2615c
Wen Wang
673d449ca2f8aff91ef2615d
Sunhan Xu
admin_assigned
2024-11-20T13:42:43.952Z
661d280abca1a6038b2bfe5e
/avatars/62b4818824c6c0ef9afad0d727618681.svg
Sunhan Xu
user
Shuanghan
673d449ca2f8aff91ef2615e
Hang Xiong
673d449ca2f8aff91ef2615f
Yun Zeng
admin_assigned
2024-11-20T13:54:47.572Z
66a0bc420b187dd45f7bf08f
/avatars/31bc135113862d932958607fdccd9c4b.svg
Yun Zeng
user
Yun0418
673d449ca2f8aff91ef26160
Jian Guo
673d449ca2f8aff91ef26161
Shuxun Wang
admin_assigned
2024-11-20T13:55:23.017Z
6459f8cac70a926dc46b10d1
/avatars/d68619fb59edd7a93a9709695b12c94d.svg
wangshuxun
user
wangshuxun
673d449ca2f8aff91ef26162
Chunfeng Yuan
673d449ca2f8aff91ef26163
Bing Li
673d449ca2f8aff91ef26164
Weiming Hu
admin_assigned
2024-11-20T13:56:57.438Z
6641c7c05aa8e7031cc6afc0
/avatars/dcf2b101a7afab828bc308f922648e3f.svg
Weiming Hu
user
huweim
SEAGULL: No-reference Image Quality Assessment for Regions of Interest
  via Vision-Language Instruction Tuning
Existing Image Quality Assessment (IQA) methods achieve remarkable success in
analyzing quality for overall image, but few works explore quality analysis for
Regions of Interest (ROIs). The quality analysis of ROIs can provide
fine-grained guidance for image quality improvement and is crucial for
scenarios focusing on region-level quality. This paper proposes a novel
network, SEAGULL, which can SEe and Assess ROIs quality with GUidance from a
Large vision-Language model. SEAGULL incorporates a vision-language model
(VLM), masks generated by Segment Anything Model (SAM) to specify ROIs, and a
meticulously designed Mask-based Feature Extractor (MFE) to extract global and
local tokens for specified ROIs, enabling accurate fine-grained IQA for ROIs.
Moreover, this paper constructs two ROI-based IQA datasets, SEAGULL-100w and
SEAGULL-3k, for training and evaluating ROI-based IQA. SEAGULL-100w comprises
about 100w synthetic distortion images with 33 million ROIs for pre-training to
improve the model's ability of regional quality perception, and SEAGULL-3k
contains about 3k authentic distortion ROIs to enhance the model's ability to
perceive real world distortions. After pre-training on SEAGULL-100w and
fine-tuning on SEAGULL-3k, SEAGULL shows remarkable performance on fine-grained
ROI quality assessment. Code and datasets are publicly available at the
https://github.com/chencn2020/Seagull.
SEAGULL, a vision-language model-based network, assesses quality for Regions of Interest using masks and a mask-based feature extractor, achieving high performance on fine-grained ROI quality assessment.
vision-language model Region of Interest ROIs Segment Anything Model SAM Mask-based Feature Extractor MFE fine-grained IQA SEAGULL-100w SEAGULL-3k regional quality perception authentic distortion ROIs