Body: <p>When designing CNN for image recogition a commonly used sainty check to see if a model is working/designed fine is to see if we are able to overfit the model with a very small subset of images.</p>
  <p>I am trying out GANs. While designing GAN I took a dataset with just one image(full black image). I used the DCGAN implementation in pytorch website<a href="https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html" rel="nofollow noreferrer">code link</a>.</p>
  <p>I tried training the model with this just one black image and even after training for 100s-1000 epochs. I am not able to overfit the model ie generate a black (or something close). All what is generated are random noise image as below<a href="https://i.sstatic.net/YdjLZ.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/YdjLZ.jpg" alt="enter image description here" /></a></p>
  <p>However the model does work well for celeba dataset(the one used in the tutorial). Which means the model is good. Can anybody help me why overfitting is very difficult/impossible when using a single image.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-07T04:03:02.217Z
Comments.Text: Complete guess here, but it might be the loss function. If the desired image is completely black perhaps all the 0 values are not giving any cost, and so the GAN never actually learns anything.
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-07T08:51:15.777Z
Comments.Text: hi @Recessive , Thanks for the response this does make a point. To validate on this I took an image from MNIST dataset and I tired overfitting the model and tried training for 1000 iterations. I was still getting Random Noise. Then I had a thought that the number of epochs (1000) were a lot less as the original MNIST had 60000 images. I trained the model with mnist image for 10000 iterations and within almost 4000 iterations it started to overfit and started mimicing my input image.  Then I went ahead and tried with the black image and it was difficult for it to mimic the black colour
Comments.UserDisplayName: ArunJose
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-07T08:57:21.480Z
Comments.Text: but however after about 10000+ epochs the noise was still present but I got an increase in pixels that were close to black.
Comments.UserDisplayName: ArunJose
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-19T11:49:04.643Z
Comments.Text: Hi, I am working on the same problem now. Did you manage to figure out the problem? Thanks
Comments.UserDisplayName: Toby V.
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-19T13:01:20.170Z
Comments.Text: Training more epochs was the solution
Comments.UserDisplayName: ArunJose
Tags: <generative-adversarial-networks><overfitting>
Title: How to overfit GANs with a single image
UserDisplayName: ArunJose
ThreadText: # How to overfit GANs with a single image
  
  (Asked by: username_0 on 2020-08-06)
  
  When designing CNN for image recogition a commonly used sainty check to see if a model is working/designed fine is to see if we are able to overfit the model with a very small subset of images.
  
  I am trying out GANs. While designing GAN I took a dataset with just one image(full black image). I used the DCGAN implementation in pytorch website[code link](https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html).
  
  I tried training the model with this just one black image and even after training for 100s-1000 epochs. I am not able to overfit the model ie generate a black (or something close). All what is generated are random noise image as below[![enter image description here](https://i.sstatic.net/YdjLZ.jpg)](https://i.sstatic.net/YdjLZ.jpg)
  
  However the model does work well for celeba dataset(the one used in the tutorial). Which means the model is good. Can anybody help me why overfitting is very difficult/impossible when using a single image.
  
  **Tags:** generative-adversarial-networks, overfitting
  
  > **Comments:**
  > **username_1**: Complete guess here, but it might be the loss function. If the desired image is completely black perhaps all the 0 values are not giving any cost, and so the GAN never actually learns anything.  
  > **username_0**: hi @username_1 , Thanks for the response this does make a point. To validate on this I took an image from MNIST dataset and I tired overfitting the model and tried training for 1000 iterations. I was still getting Random Noise. Then I had a thought that the number of epochs (1000) were a lot less as the original MNIST had 60000 images. I trained the model with mnist image for 10000 iterations and within almost 4000 iterations it started to overfit and started mimicing my input image.  Then I went ahead and tried with the black image and it was difficult for it to mimic the black colour  
  > **username_0**: but however after about 10000+ epochs the noise was still present but I got an increase in pixels that were close to black.  
  > **username_2**: Hi, I am working on the same problem now. Did you manage to figure out the problem? Thanks  
  > **username_0**: Training more epochs was the solution  
  
  ---
  
  ## 0 answers
  
  