Answers.Body: <p>Images such as this one are produced using <a href="https://en.wikipedia.org/wiki/Generative_adversarial_network" rel="nofollow noreferrer">generative adversial network</a>, which is build from two models:</p>
  <ul>
  <li>one to generate images given a random vector as input</li>
  <li>another trying to detect the generated image from two images, with one of them being real</li>
  </ul>
  <p>Then the weights of the first model are updated if the second one detected which image is artificial, and the second model is updated if its prediction is wrong.</p>
  <p>Of course you might build a model that can sometime detect AI generated images, but it is probably not possible to differentiate them all the time. Then, if you build such model that is better than any other model to detect generated images, it is possible to create another model trained to fool it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-13T07:13:54.710Z
Answers.LastActivityDate: 2022-07-13T07:13:54.710Z
Answers.UserDisplayName: theophile
Answers.Body: <p>I am not an expert, but it feels like these GANs are not paying attention to the clothes and the background and make them &quot;fluid&quot;.</p>
  <p>Like, what is this hat the woman in your example is wearing? Why is the right side of the background looks like it is a mix of liquid paint?</p>
  <p>Or here:
  <a href="https://i.sstatic.net/LBZK6.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/LBZK6.jpg" alt="enter image description here" /></a>
  What is she wearing? Did she kill a rat to make these clothes? And similar fluid background.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-13T10:44:16.343Z
Answers.LastActivityDate: 2022-07-13T10:44:16.343Z
Answers.UserDisplayName: asdfasdfsdf
Answers.Body: <p><strong>TL;DR: Yes, but it's becoming more and more difficult, even for humans, as generative models get better and better. It's a quite hot research topic.</strong></p>
  <p><em>Disclaimer: I am not affiliated with any of the authors, I'm just studying this research topic.</em></p>
  <p>Humans usually look for some visual artifacts (as all the other answers point out), for example</p>
  <ul>
  <li>Colour or texture artifacts (colour blobs, unrealistic texture)</li>
  <li>Asymmetries or inconsistencies in the image (this is easy to spot in faces or hair, for example)</li>
  <li>Anomalies in color, lighting, image parts</li>
  </ul>
  <p><a href="https://i.sstatic.net/2EcVj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/2EcVj.png" alt="image anomalies" /></a></p>
  <p>But as models become more and more advanced, these artifacts are becoming harder to spot, if not completely disappeared. As in 2023, images coming from diffusion models like Stable Diffusion or Midjourney API have a photorealistic quality, and often are already indistinguishable from real images (see some examples <a href="https://www.midjourney.com/showcase/recent/" rel="nofollow noreferrer">here</a>).</p>
  <p>For these reasons, we want to find automatic and more robust detection approaches to prevent malicious uses of these AI generation models.</p>
  <h2>Detection approach</h2>
  <p>A simple approach is to train a detector on AI generated images, which classifies the image as real or AI generated either by looking at the whole images or at single patches [Chai et al., 2020].</p>
  <p>A more performing approach is to exploit some invisible artifacts created by the convolutional upsampling, which is commonly used in GANs and in <em>some</em> Diffusion models (Stable Diffusion, for example) to create high-resolution images. While invisible in the image domain, this trace can be easily extracted and identified in the frequency domain [Marra et al., 2019, Yu et al., 2019].</p>
  <p><a href="https://i.sstatic.net/KAw9O.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/KAw9O.png" alt="Frequency domain artifacts" /></a></p>
  <p>The detector needs to be robust to common image modifications (contrast/luminosity, colour jittering, jpeg compression,etc.) and also to adversarial attacks on the detector.</p>
  <h2>Watermarking/Fingerprinting</h2>
  <p>To proactively improve the detection performance, the developers of generative models could include a <em>watermark</em> in their images, to mark the images produced by their models as AI-generated. This watermark is typically invisible, and can be generated in several ways:</p>
  <ul>
  <li>Traditional approaches are based on frequency decompositions of the image, constructed through DCT, DWT, Fourier-Mellin, or complex wavelet transformations Cox et al., 1996, O’ Ruanaidh et al., 1996,
  O’Ruanaidh and Pun, 1997]. These frequency transformations all share the beneficial property that simple image manipulations, such as translations, rotations, and resizing are easily understandable and watermarks can be constructed with robustness to these transformations in mind.</li>
  <li>Model-based approaches use a different learned model to embed a watermark in the image. Hayes and Danezis [2017] and Zhu
  et al. [2018] propose strategies to learn watermarking end-to-end, where both the watermark encoder and the watermark decoder are learned models, optimized via adversarial objectives to maximize transmission and robustness [Zhang et al., 2019]. Zeng et al. [2023] present a related approach, in which a neural network watermarked encoder and its associate detector are jointly learned using an image dataset. Notably these approaches still work like a traditional watermark in that the encoder imprints a post-hoc signal onto a given image - however the type of imprint is now learned.</li>
  <li>More recent approaches use another model or the generative model itself to either embed the watermark after the generation process or during the generation process. Some approaches consist in embedding a watermark in training data [Yu et al., 2022], in some components of the model (e.g the convolutional decoder) [Fernandez et al., 2023], or by slightly modifying the distribution from where sampling is performed [Wen et al., 2023].</li>
  </ul>
  <p>As detectors, watermarks need to be robust to common image transformations (such as cropping, contrast/luminance editing, jpeg compression, etc.) and adversarial attacks that actively try to remove the watermark.</p>
  <h2>References</h2>
  <ul>
  <li>Gragnaniello, Diego et al. “Are GAN Generated Images Easy to Detect? A Critical Analysis of the State-Of-The-Art.” 2021 IEEE International Conference on Multimedia and Expo (ICME) (2021): 1-6.</li>
  <li>enbo Wan, Jun Wang, Yunming Zhang, Jing Li, Hui Yu, and Jiande Sun. A comprehensive survey on
  robust image watermarking. Neurocomputing, 488:226–247, June 2022. ISSN 0925-2312. doi: 10.
  1016/j.neucom.2022.02.083. URL <a href="https://www.sciencedirect.com/science/article/" rel="nofollow noreferrer">https://www.sciencedirect.com/science/article/</a>
  pii/S0925231222002533.</li>
  <li>[Cox et al., 1996] Cox, I., Kilian, J., Leighton, T., and Shamoon, T. (1996). Secure spread spectrum watermarking for images, audio and video. In Proceedings of 3rd IEEE International Conference on Image Processing, volume 3, pages 243–246 vol.3.</li>
  <li>[O’ Ruanaidh et al., 1996] O’ Ruanaidh, J., Dowling, W., and Boland, F. (1996). Watermarking digital
  images for copyright protection. IEE PROCEEDINGS VISION IMAGE AND SIGNAL PROCESSING, 143:250–256.</li>
  <li>O’Ruanaidh and Pun, 1997] O’Ruanaidh, J. J. and Pun, T. (1997). Rotation, scale and translation invariant digital image watermarking. In Proceedings of International Conference on Image Processing, volume 1, pages 536–539. IEEE.</li>
  <li>[Zhu et al., 2018] Zhu, J., Kaplan, R., Johnson, J., and Fei-Fei, L. (2018). Hidden: Hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV), pages 657–672. 3</li>
  <li>[Marra et al., 2019] Marra, F., Gragnaniello, D., Verdoliva, L., and Poggi, G. (2019). Do gans leave artificial fingerprints? In 2019 IEEE conference on multimedia information processing and retrieval
  (MIPR), pages 506–511. IEEE.</li>
  <li>[Yu et al., 2019] Yu, N., Davis, L. S., and Fritz, M. (2019). Attributing fake images to gans: Learning and analyzing gan fingerprints. In Proceedings of the IEEE/CVF international conference on
  computer vision, pages 7556–7566.
  2</li>
  <li>[Chai et al., 2020] Chai, L., Bau, D., Lim, S.-N., and Isola, P. (2020). What makes fake images detectable? understanding properties that generalize</li>
  <li>[Yu et al., 2022] Yu, N., Skripniuk, V., Abdelnabi, S., and Fritz, M. (2022). Artificial fingerprinting for generative models: Rooting deepfake attribution in training data.</li>
  <li>[Corvi et al., 2023] Corvi, R., Cozzolino, D., Zingarini, G., Poggi, G., Nagano, K., and Verdoliva, L.
  (2023). On the detection of synthetic images generated by diffusion models. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE.</li>
  <li>[Wen et al., 2023] Wen, Y., Kirchenbauer, J., Geiping, J., and Goldstein, T. (2023). Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust.</li>
  <li>[Fernandez et al., 2023] Fernandez, P., Couairon, G., J ́egou, H., Douze, M., and Furon, T. (2023). The stable signature: Rooting watermarks in latent diffusion models.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-08T15:04:05.787Z
Answers.LastActivityDate: 2023-07-08T15:04:05.787Z
Answers.UserDisplayName: Ciodar
Answers.Body: <p>There's a paper that claims to detect AI generated images with a 95% accuracy.
  <a href="https://www.researchgate.net/publication/326053461_Detection_of_GAN-Generated_Fake_Images_over_Social_Networks" rel="nofollow noreferrer">https://www.researchgate.net/publication/326053461_Detection_of_GAN-Generated_Fake_Images_over_Social_Networks</a></p>
  <p>A search with the right keywords can reveal more such research.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-28T06:30:25.900Z
Answers.LastActivityDate: 2023-05-28T06:30:25.900Z
Answers.UserDisplayName: Nav
Answers.Body: <p>I have not worked practically with GANs and just know their theory, but I do not agree 100% with this comment that AI chooses stupid things for clothes or backgrounds. I remember it could be detected when a video was generated with deep learning methods from Obama.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-13T14:25:26.797Z
Answers.Text: You are supposed to use comments for this.
Answers.UserDisplayName: asdfasdfsdf
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-13T14:29:29.533Z
Answers.Text: I wanted to do that but I can not comment. it is restricted
Answers.UserDisplayName: Pouyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-13T19:27:07.197Z
Answers.Text: It is better to wait a little then. Maybe a little frustrating, but you risk getting downvotes, which will make your rep go down, if you try and work around the way the tools are set up. Sadly it has to be set up like this, because spambots would overwhelm things (costing extra volunteer moderator time) if there wasn't some significant barrier to commenting from a new account.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-13T14:23:03.290Z
Answers.LastActivityDate: 2022-07-13T14:23:03.290Z
Answers.UserDisplayName: Pouyan
Body: <p>AIs are getting better and better at creating images and art. Some of the stuff is almost impossible to be detected by the naked eye. But what about programs and algorithms? Instead of creating an image, can anything detect that this image was created by an AI?</p>
  <p>Take this one for example:</p>
  <p>This picture of a woman's face was generated by AI
  <a href="https://i.sstatic.net/S0WyG.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/S0WyG.jpg" alt="This picture of a woman's face was generated by AI" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-28T09:17:59.763Z
Comments.Text: Once we work with bit arrays or matrices after discretization, things become difficult. The artificial bitstream, either coming from a natural or artificial source at the origin, is identical. Take a simple "hello world" as an example. 11 characters, at say 8bits/char, 88 bits. But all that information encodes the text content only, no bit encodes the natural/artificial origin. Therefore, how can we tell if this was produced by a human typing on a keyboard or a code program? That information is by definition lost in discretization.
Comments.UserDisplayName: Jaume Oliver Lafont
Tags: <image-recognition><image-processing><human-like><face-recognition><face-detection>
Title: Can an AI generated image (such as pic of human face) be detected that it's AI generated?
UserDisplayName: No Name
ThreadText: # Can an AI generated image (such as pic of human face) be detected that it's AI generated?
  
  (Asked by: username_0 on 2022-07-13)
  
  AIs are getting better and better at creating images and art. Some of the stuff is almost impossible to be detected by the naked eye. But what about programs and algorithms? Instead of creating an image, can anything detect that this image was created by an AI?
  
  Take this one for example:
  
  This picture of a woman's face was generated by AI
  [![This picture of a woman's face was generated by AI](https://i.sstatic.net/S0WyG.jpg)](https://i.sstatic.net/S0WyG.jpg)
  
  **Tags:** image-recognition, image-processing, human-like, face-recognition, face-detection
  > **Comments:**
  > **username_1**: Once we work with bit arrays or matrices after discretization, things become difficult. The artificial bitstream, either coming from a natural or artificial source at the origin, is identical. Take a simple "hello world" as an example. 11 characters, at say 8bits/char, 88 bits. But all that information encodes the text content only, no bit encodes the natural/artificial origin. Therefore, how can we tell if this was produced by a human typing on a keyboard or a code program? That information is by definition lost in discretization.  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_2 on 2022-07-13. Score: 1
  
  Images such as this one are produced using [generative adversial network](https://en.wikipedia.org/wiki/Generative_adversarial_network), which is build from two models:
  
  * one to generate images given a random vector as input
  * another trying to detect the generated image from two images, with one of them being real
  
  Then the weights of the first model are updated if the second one detected which image is artificial, and the second model is updated if its prediction is wrong.
  
  Of course you might build a model that can sometime detect AI generated images, but it is probably not possible to differentiate them all the time. Then, if you build such model that is better than any other model to detect generated images, it is possible to create another model trained to fool it.
  
  
  ---
  
  ### Answer by username_3 on 2022-07-13. Score: 1
  
  I am not an expert, but it feels like these GANs are not paying attention to the clothes and the background and make them "fluid".
  
  Like, what is this hat the woman in your example is wearing? Why is the right side of the background looks like it is a mix of liquid paint?
  
  Or here:
  [![enter image description here](https://i.sstatic.net/LBZK6.jpg)](https://i.sstatic.net/LBZK6.jpg)
  What is she wearing? Did she kill a rat to make these clothes? And similar fluid background.
  
  
  ---
  
  ### Answer by username_4 on 2023-07-08. Score: 1
  
  **TL;DR: Yes, but it's becoming more and more difficult, even for humans, as generative models get better and better. It's a quite hot research topic.**
  
  *Disclaimer: I am not affiliated with any of the authors, I'm just studying this research topic.*
  
  Humans usually look for some visual artifacts (as all the other answers point out), for example
  
  * Colour or texture artifacts (colour blobs, unrealistic texture)
  * Asymmetries or inconsistencies in the image (this is easy to spot in faces or hair, for example)
  * Anomalies in color, lighting, image parts
  
  [![image anomalies](https://i.sstatic.net/2EcVj.png)](https://i.sstatic.net/2EcVj.png)
  
  But as models become more and more advanced, these artifacts are becoming harder to spot, if not completely disappeared. As in 2023, images coming from diffusion models like Stable Diffusion or Midjourney API have a photorealistic quality, and often are already indistinguishable from real images (see some examples [here](https://www.midjourney.com/showcase/recent/)).
  
  For these reasons, we want to find automatic and more robust detection approaches to prevent malicious uses of these AI generation models.
  
  ## Detection approach
  
  A simple approach is to train a detector on AI generated images, which classifies the image as real or AI generated either by looking at the whole images or at single patches [Chai et al., 2020].
  
  A more performing approach is to exploit some invisible artifacts created by the convolutional upsampling, which is commonly used in GANs and in *some* Diffusion models (Stable Diffusion, for example) to create high-resolution images. While invisible in the image domain, this trace can be easily extracted and identified in the frequency domain [Marra et al., 2019, Yu et al., 2019].
  
  [![Frequency domain artifacts](https://i.sstatic.net/KAw9O.png)](https://i.sstatic.net/KAw9O.png)
  
  The detector needs to be robust to common image modifications (contrast/luminosity, colour jittering, jpeg compression,etc.) and also to adversarial attacks on the detector.
  
  ## Watermarking/Fingerprinting
  
  To proactively improve the detection performance, the developers of generative models could include a *watermark* in their images, to mark the images produced by their models as AI-generated. This watermark is typically invisible, and can be generated in several ways:
  
  * Traditional approaches are based on frequency decompositions of the image, constructed through DCT, DWT, Fourier-Mellin, or complex wavelet transformations Cox et al., 1996, O’ Ruanaidh et al., 1996,
    O’Ruanaidh and Pun, 1997]. These frequency transformations all share the beneficial property that simple image manipulations, such as translations, rotations, and resizing are easily understandable and watermarks can be constructed with robustness to these transformations in mind.
  * Model-based approaches use a different learned model to embed a watermark in the image. Hayes and Danezis [2017] and Zhu
    et al. [2018] propose strategies to learn watermarking end-to-end, where both the watermark encoder and the watermark decoder are learned models, optimized via adversarial objectives to maximize transmission and robustness [Zhang et al., 2019]. Zeng et al. [2023] present a related approach, in which a neural network watermarked encoder and its associate detector are jointly learned using an image dataset. Notably these approaches still work like a traditional watermark in that the encoder imprints a post-hoc signal onto a given image - however the type of imprint is now learned.
  * More recent approaches use another model or the generative model itself to either embed the watermark after the generation process or during the generation process. Some approaches consist in embedding a watermark in training data [Yu et al., 2022], in some components of the model (e.g the convolutional decoder) [Fernandez et al., 2023], or by slightly modifying the distribution from where sampling is performed [Wen et al., 2023].
  
  As detectors, watermarks need to be robust to common image transformations (such as cropping, contrast/luminance editing, jpeg compression, etc.) and adversarial attacks that actively try to remove the watermark.
  
  ## References
  
  * Gragnaniello, Diego et al. “Are GAN Generated Images Easy to Detect? A Critical Analysis of the State-Of-The-Art.” 2021 IEEE International Conference on Multimedia and Expo (ICME) (2021): 1-6.
  * enbo Wan, Jun Wang, Yunming Zhang, Jing Li, Hui Yu, and Jiande Sun. A comprehensive survey on
    robust image watermarking. Neurocomputing, 488:226–247, June 2022. ISSN 0925-2312. doi: 10.
    1016/j.neucom.2022.02.083. URL <https://www.sciencedirect.com/science/article/>
    pii/S0925231222002533.
  * [Cox et al., 1996] Cox, I., Kilian, J., Leighton, T., and Shamoon, T. (1996). Secure spread spectrum watermarking for images, audio and video. In Proceedings of 3rd IEEE International Conference on Image Processing, volume 3, pages 243–246 vol.3.
  * [O’ Ruanaidh et al., 1996] O’ Ruanaidh, J., Dowling, W., and Boland, F. (1996). Watermarking digital
    images for copyright protection. IEE PROCEEDINGS VISION IMAGE AND SIGNAL PROCESSING, 143:250–256.
  * O’Ruanaidh and Pun, 1997] O’Ruanaidh, J. J. and Pun, T. (1997). Rotation, scale and translation invariant digital image watermarking. In Proceedings of International Conference on Image Processing, volume 1, pages 536–539. IEEE.
  * [Zhu et al., 2018] Zhu, J., Kaplan, R., Johnson, J., and Fei-Fei, L. (2018). Hidden: Hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV), pages 657–672. 3
  * [Marra et al., 2019] Marra, F., Gragnaniello, D., Verdoliva, L., and Poggi, G. (2019). Do gans leave artificial fingerprints? In 2019 IEEE conference on multimedia information processing and retrieval
    (MIPR), pages 506–511. IEEE.
  * [Yu et al., 2019] Yu, N., Davis, L. S., and Fritz, M. (2019). Attributing fake images to gans: Learning and analyzing gan fingerprints. In Proceedings of the IEEE/CVF international conference on
    computer vision, pages 7556–7566.
    2
  * [Chai et al., 2020] Chai, L., Bau, D., Lim, S.-N., and Isola, P. (2020). What makes fake images detectable? understanding properties that generalize
  * [Yu et al., 2022] Yu, N., Skripniuk, V., Abdelnabi, S., and Fritz, M. (2022). Artificial fingerprinting for generative models: Rooting deepfake attribution in training data.
  * [Corvi et al., 2023] Corvi, R., Cozzolino, D., Zingarini, G., Poggi, G., Nagano, K., and Verdoliva, L.
    (2023). On the detection of synthetic images generated by diffusion models. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE.
  * [Wen et al., 2023] Wen, Y., Kirchenbauer, J., Geiping, J., and Goldstein, T. (2023). Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust.
  * [Fernandez et al., 2023] Fernandez, P., Couairon, G., J ́egou, H., Douze, M., and Furon, T. (2023). The stable signature: Rooting watermarks in latent diffusion models.
  
  
  ---
  
  ### Answer by username_5 on 2023-05-28. Score: 0
  
  There's a paper that claims to detect AI generated images with a 95% accuracy.
  <https://www.researchgate.net/publication/326053461_Detection_of_GAN-Generated_Fake_Images_over_Social_Networks>
  
  A search with the right keywords can reveal more such research.
  
  
  ---
  
  ### Answer by username_6 on 2022-07-13. Score: 0
  
  I have not worked practically with GANs and just know their theory, but I do not agree 100% with this comment that AI chooses stupid things for clothes or backgrounds. I remember it could be detected when a video was generated with deep learning methods from Obama.
  
  > **Comments:**
  > **username_3**: You are supposed to use comments for this.  
  > **username_6**: I wanted to do that but I can not comment. it is restricted  
  > **username_7**: It is better to wait a little then. Maybe a little frustrating, but you risk getting downvotes, which will make your rep go down, if you try and work around the way the tools are set up. Sadly it has to be set up like this, because spambots would overwhelm things (costing extra volunteer moderator time) if there wasn't some significant barrier to commenting from a new account.  
  
  ---
  
  