Answers.Body: <p>You could add the desired traffic sign location to the latent vector and then arrange that the generator incurs loss if the traffic sign is not at the right place in the generated image. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T12:54:23.453Z
Answers.Text: If I knew position of traffic sign in the generated image, I could have just use that position as annotation of image. But since it's generated image based on let's say 1000 real images, I down know position of traffic sign in the generated image.
Answers.UserDisplayName: kocica
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T13:34:44.457Z
Answers.Text: The sign location could be random and the generator's task would be to generate the sign at that particular location. This would be similar to a conditional GAN: instead of the desired class label (CGAN), here you would  supply the desired object location.
Answers.UserDisplayName: ssegvic
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-01T19:57:43.140Z
Answers.Text: Thats nice idea, but still the YOLO detection network learns (besides others) postion of the object in the full image. That means if all the images in the training dataset would have traffic sign on the same place, YOLO would learn that feature and then, most-likely, would detect traffic signs in that desired location _only_ (but I am not entirely sure with that tho). I will give it a try and we shall see :)
Answers.UserDisplayName: kocica
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T11:57:59.670Z
Answers.LastActivityDate: 2019-01-31T11:57:59.670Z
Answers.UserDisplayName: ssegvic
Answers.Body: <p>I think you'll <a href="https://machinelearning.apple.com/2017/07/07/GAN.html" rel="nofollow noreferrer">enjoy this work from Apple</a> on improving the realism of synthetic images.  Essentially what you need to do is generate a synthetic image <em>then</em> have your GAN <em>modify</em> the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this loss function is going to take a little work!  </p>
  
  <p>Making synthetic data realistic enough to allow models to generalize successfully in the real world is a very active and exciting area of research, not least with respect to robotics, and so the work you are doing now should make you very attractive indeed to the right employer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T13:08:33.977Z
Answers.Text: That's amazing!
Answers.UserDisplayName: kocica
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T14:09:13.550Z
Answers.Text: Glad you liked it!  I would be most interested in reading your thesis when you are finished - I think there is a good chance that lots of other folk will want to read it too.
Answers.UserDisplayName: Edward Dixon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T20:24:07.173Z
Answers.Text: Indeed, I think these dataset generative tools/papers would help a lot of guys at least training some nets. Unfortunately after reading work from Apple Iam not sure I can do such complex system in just few months but I'll do my best and let you know :-)
Answers.UserDisplayName: kocica
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T22:45:10.717Z
Answers.Text: A Floridian MSc student by the name of Connor Shorten - active on Twitter, Medium, etc. - has very similar interests.  Probably you two should have a Skype call.  I think you under-estimate yourself (and also, since the Apple work, there'll be many others on the same road).
Answers.UserDisplayName: Edward Dixon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-31T13:03:24.213Z
Answers.LastActivityDate: 2019-01-31T13:03:24.213Z
Answers.UserDisplayName: Edward Dixon
Body: <p>I'm doing bachaleor thesis on traffic sign detection using single shot detector called YOLO. These single shot detectors can perform detection of objects in image and so they have specific way of training, ie. training on full images. Thats quite problem for me, because the biggest real dataset with full traffic sign images is <a href="https://btsd.ethz.ch/shareddata/" rel="nofollow noreferrer">Belgian one</a> with 9000 images in 210 classes, which is unfortunately not enough to train good detector.</p>
  
  <p>To overcome this problem, I've created <a href="https://github.com/kocica/DatasetGenerator" rel="nofollow noreferrer">DatasetGenerator</a>, which does quite good job in generating synthetic datasets, you can see in the <a href="https://github.com/kocica/DatasetGenerator/tree/master/results" rel="nofollow noreferrer">results directory</a>.</p>
  
  <p>Recently I came across GAN's which can (besides others) generate or extend existing dataset and I would like to use these networks to compare with my dataset generator. I've tried <a href="https://medium.com/ai-society/gans-from-scratch-1-a-deep-introduction-with-code-in-pytorch-and-tensorflow-cb03cdcdba0f" rel="nofollow noreferrer">this introduction to GANs</a> succesfully.</p>
  
  <p>The problem is it's unsupervised learning and so there are no annotations. It means it's able to extend my dataset of traffic signs, but the generated dataset won't be annotated at all, which is problem.</p>
  
  <p>So my question is: Is there any way how to use GAN's to extend my dataset of full traffic sign images with annotations of traffic sign class and position? Actually the class is not important, because I can do it separately for each class, but what matters is the position of traffic sign in generated image.</p>
  
Tags: <convolutional-neural-networks><datasets><generative-adversarial-networks>
Title: Using GAN's to generate dataset for CNN training
UserDisplayName: kocica
ThreadText: # Using GAN's to generate dataset for CNN training
  
  (Asked by: username_0 on 2019-01-31)
  
  I'm doing bachaleor thesis on traffic sign detection using single shot detector called YOLO. These single shot detectors can perform detection of objects in image and so they have specific way of training, ie. training on full images. Thats quite problem for me, because the biggest real dataset with full traffic sign images is [Belgian one](https://btsd.ethz.ch/shareddata/) with 9000 images in 210 classes, which is unfortunately not enough to train good detector.
  
  To overcome this problem, I've created [DatasetGenerator](https://github.com/kocica/DatasetGenerator), which does quite good job in generating synthetic datasets, you can see in the [results directory](https://github.com/kocica/DatasetGenerator/tree/master/results).
  
  Recently I came across GAN's which can (besides others) generate or extend existing dataset and I would like to use these networks to compare with my dataset generator. I've tried [this introduction to GANs](https://medium.com/ai-society/gans-from-scratch-1-a-deep-introduction-with-code-in-pytorch-and-tensorflow-cb03cdcdba0f) succesfully.
  
  The problem is it's unsupervised learning and so there are no annotations. It means it's able to extend my dataset of traffic signs, but the generated dataset won't be annotated at all, which is problem.
  
  So my question is: Is there any way how to use GAN's to extend my dataset of full traffic sign images with annotations of traffic sign class and position? Actually the class is not important, because I can do it separately for each class, but what matters is the position of traffic sign in generated image.
  
  **Tags:** convolutional-neural-networks, datasets, generative-adversarial-networks
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-01-31. Score: 3 | Accepted answer
  
  I think you'll [enjoy this work from Apple](https://machinelearning.apple.com/2017/07/07/GAN.html) on improving the realism of synthetic images. Essentially what you need to do is generate a synthetic image *then* have your GAN *modify* the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this loss function is going to take a little work!
  
  Making synthetic data realistic enough to allow models to generalize successfully in the real world is a very active and exciting area of research, not least with respect to robotics, and so the work you are doing now should make you very attractive indeed to the right employer.
  
  > **Comments:**
  > **username_0**: That's amazing!  
  > **username_2**: Glad you liked it!  I would be most interested in reading your thesis when you are finished - I think there is a good chance that lots of other folk will want to read it too.  
  > **username_0**: Indeed, I think these dataset generative tools/papers would help a lot of guys at least training some nets. Unfortunately after reading work from Apple Iam not sure I can do such complex system in just few months but I'll do my best and let you know :-)  
  > **username_2**: A Floridian MSc student by the name of Connor Shorten - active on Twitter, Medium, etc. - has very similar interests.  Probably you two should have a Skype call.  I think you under-estimate yourself (and also, since the Apple work, there'll be many others on the same road).  
  
  ---
  
  ### Answer by username_1 on 2019-01-31. Score: 0
  
  You could add the desired traffic sign location to the latent vector and then arrange that the generator incurs loss if the traffic sign is not at the right place in the generated image.
  
  > **Comments:**
  > **username_0**: If I knew position of traffic sign in the generated image, I could have just use that position as annotation of image. But since it's generated image based on let's say 1000 real images, I down know position of traffic sign in the generated image.  
  > **username_1**: The sign location could be random and the generator's task would be to generate the sign at that particular location. This would be similar to a conditional GAN: instead of the desired class label (CGAN), here you would  supply the desired object location.  
  > **username_0**: Thats nice idea, but still the YOLO detection network learns (besides others) postion of the object in the full image. That means if all the images in the training dataset would have traffic sign on the same place, YOLO would learn that feature and then, most-likely, would detect traffic signs in that desired location _only_ (but I am not entirely sure with that tho). I will give it a try and we shall see :)  
  
  ---
  
  