Detecting AI-Generated Images via CLIP
<|reference_start|>Detecting AI-Generated Images via CLIP: As AI-generated image (AIGI) methods become more powerful and accessible, it has become a critical task to determine if an image is real or AI-generated. Because AIGI lack the signatures of photographs and have their own unique patterns, new models are needed to determine if an image is AI-generated. In this paper, we investigate the ability of the Contrastive Language-Image Pre-training (CLIP) architecture, pre-trained on massive internet-scale data sets, to perform this differentiation. We fine-tune CLIP on real images and AIGI from several generative models, enabling CLIP to determine if an image is AI-generated and, if so, determine what generation method was used to create it. We show that the fine-tuned CLIP architecture is able to differentiate AIGI as well or better than models whose architecture is specifically designed to detect AIGI. Our method will significantly increase access to AIGI-detecting tools and reduce the negative effects of AIGI on society, as our CLIP fine-tuning procedures require no architecture changes from publicly available model repositories and consume significantly less GPU resources than other AIGI detection models.<|reference_end|>
@article{moskowitz2024detecting,
    title={Detecting AI-Generated Images via CLIP},
    author={A.G. Moskowitz, T. Gaona, J. Peterson},
    journal={arXiv preprint arXiv:2404.08788},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.08788},
    primaryClass={cs.CV cs.LG}
}