QPT V2: Masked Image Modeling Advances Visual Scoring
<|reference_start|>QPT V2: Masked Image Modeling Advances Visual Scoring: Quality assessment and aesthetics assessment aim to evaluate the perceived quality and aesthetics of visual content. Current learning-based methods suffer greatly from the scarcity of labeled data and usually perform sub-optimally in terms of generalization. Although masked image modeling (MIM) has achieved noteworthy advancements across various high-level tasks (e.g., classification, detection etc.). In this work, we take on a novel perspective to investigate its capabilities in terms of quality- and aesthetics-awareness. To this end, we propose Quality- and aesthetics-aware pretraining (QPT V2), the first pretraining framework based on MIM that offers a unified solution to quality and aesthetics assessment. To perceive the high-level semantics and fine-grained details, pretraining data is curated. To comprehensively encompass quality- and aesthetics-related factors, degradation is introduced. To capture multi-scale quality and aesthetic information, model structure is modified. Extensive experimental results on 11 downstream benchmarks clearly show the superior performance of QPT V2 in comparison with current state-of-the-art approaches and other pretraining paradigms. Code and models will be released at \url{https://github.com/KeiChiTse/QPT-V2}.<|reference_end|>
@article{xie2024qpt,
    title={QPT V2: Masked Image Modeling Advances Visual Scoring},
    author={Qizhi Xie, Kun Yuan, Yunpeng Qu, Mingda Wu, Ming Sun, Chao Zhou,
  Jihong Zhu},
    journal={arXiv preprint arXiv:2407.16541},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.16541},
    primaryClass={cs.CV cs.MM}
}