Answers.Body: <p>Not necessarily it depends on the function of the problem space for both the GANs.</p>
  <p>A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be a better pitcher because the threshold of reaction time.</p>
  <p>We don't yet know if a maximum threshold on realistic fake image generation exists or if a threshold on detection exists.</p>
  <p>As both reach near perfect accuracy it could be that the amount of nodes needed to detect a nearly perfect generated image from real image is more neurons than atoms in the universe, or conversely the amount of nodes needed to generated a nearly perfect image could reach impossible proportion we won't know until we continue to make better and better networks that close in on the boundary of generation and detection of <strong>real</strong> vs <strong>fake</strong> images from a neural network.</p>
  <p>Edit:
  Let's imagine this problem. One adversary edits an image with a colored line of pixels the goal hide the line by editing the image, the student is responsible with finding the line after the adversary changes the image. The problem can become <strong>infinitely difficult</strong> change all pixels to be the color of the line. The line is impossible to find the adversary always wins, <strong>if</strong> it finds this solution meaning it is in its reachable problem space based on its hardware capabilities and its learning model. Which we should assume it would because it's such a simple task.</p>
  <p>Deep fake detection is not bound to fail because the limit on the effectiveness of a generative model <strong>may</strong> have a steeper limit than a discriminator at near optimal performance. I have not seen any paper about this specifically and in fact I believe the discriminator has a more difficult job in most cases I just disagree with deciding at this moment that the detectors are doomed. The combination of creating real images motion and sound perfectly in sink is not a trivial problem, in some scenarios it is basically impossible.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T22:14:25.623Z
Answers.Text: Thanks for your contribution. However your batter / pitcher comparison seems a bit off as those actions require very different qualities (power versus reaction time... as far as I understand that sport). Do you have any source on why this kind of asymetry would arise between adversarial models ? (which I somewhat expect to perform similarly given comparable ressources - architecture, computation power ... etc.)
Answers.UserDisplayName: Lucas Morin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-14T22:48:47.890Z
Answers.Text: Good point about the cat-and-mouse game that is anything involving misuse of new technology and measures to counteract.  Hope to see more of you on SE:AI!
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-15T00:26:23.763Z
Answers.Text: @DukeZhou Thanks, is there any topic specifically in SE:AI you'd want me to look out for? Or to look into?
Answers.UserDisplayName: Michael Hearn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T20:05:29.650Z
Answers.LastActivityDate: 2022-11-15T19:02:39.237Z
Answers.LastEditDate: 2022-11-15T19:02:39.237Z
Answers.UserDisplayName: Michael Hearn
Body: <p>Deep fakes are a growing concern: the ability to credibly alter a video may have great (negative) impacts on our society. It is so much of a concern, that the biggest tech companies launched a specific challenge: <a href="https://deepfakedetectionchallenge.ai/" rel="noreferrer">https://deepfakedetectionchallenge.ai/</a>.</p>
  
  <p>However, from what I understand, most deep fake generation techniques rely on the use of adversarial models. One model generates a new image, while another model tries to detect if the image is doctored or not. Both models "learn" from being confronted with the other. </p>
  
  <p>That being said, if a good deep fake detection model emerges (from the previous challenge, or not), wouldn't it be rendered useless almost instantly by learning from it in an adversarial setting?</p>
  
Tags: <generative-adversarial-networks><adversarial-ml><deepfakes>
Title: Isn't deep fake detection bound to fail?
UserDisplayName: Lucas Morin
ThreadText: # Isn't deep fake detection bound to fail?
  
  (Asked by: username_0 on 2019-11-13)
  
  Deep fakes are a growing concern: the ability to credibly alter a video may have great (negative) impacts on our society. It is so much of a concern, that the biggest tech companies launched a specific challenge: <https://deepfakedetectionchallenge.ai/>.
  
  However, from what I understand, most deep fake generation techniques rely on the use of adversarial models. One model generates a new image, while another model tries to detect if the image is doctored or not. Both models "learn" from being confronted with the other.
  
  That being said, if a good deep fake detection model emerges (from the previous challenge, or not), wouldn't it be rendered useless almost instantly by learning from it in an adversarial setting?
  
  **Tags:** generative-adversarial-networks, adversarial-ml, deepfakes
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-13. Score: 5 | Accepted answer
  
  Not necessarily it depends on the function of the problem space for both the GANs.
  
  A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be a better pitcher because the threshold of reaction time.
  
  We don't yet know if a maximum threshold on realistic fake image generation exists or if a threshold on detection exists.
  
  As both reach near perfect accuracy it could be that the amount of nodes needed to detect a nearly perfect generated image from real image is more neurons than atoms in the universe, or conversely the amount of nodes needed to generated a nearly perfect image could reach impossible proportion we won't know until we continue to make better and better networks that close in on the boundary of generation and detection of **real** vs **fake** images from a neural network.
  
  Edit:
  Let's imagine this problem. One adversary edits an image with a colored line of pixels the goal hide the line by editing the image, the student is responsible with finding the line after the adversary changes the image. The problem can become **infinitely difficult** change all pixels to be the color of the line. The line is impossible to find the adversary always wins, **if** it finds this solution meaning it is in its reachable problem space based on its hardware capabilities and its learning model. Which we should assume it would because it's such a simple task.
  
  Deep fake detection is not bound to fail because the limit on the effectiveness of a generative model **may** have a steeper limit than a discriminator at near optimal performance. I have not seen any paper about this specifically and in fact I believe the discriminator has a more difficult job in most cases I just disagree with deciding at this moment that the detectors are doomed. The combination of creating real images motion and sound perfectly in sink is not a trivial problem, in some scenarios it is basically impossible.
  
  > **Comments:**
  > **username_0**: Thanks for your contribution. However your batter / pitcher comparison seems a bit off as those actions require very different qualities (power versus reaction time... as far as I understand that sport). Do you have any source on why this kind of asymetry would arise between adversarial models ? (which I somewhat expect to perform similarly given comparable ressources - architecture, computation power ... etc.)  
  > **username_2**: Good point about the cat-and-mouse game that is anything involving misuse of new technology and measures to counteract.  Hope to see more of you on SE:AI!  
  > **username_1**: @DukeZhou Thanks, is there any topic specifically in SE:AI you'd want me to look out for? Or to look into?  
  
  ---
  
  