Answers.Body: <p>There is no golden rule on number of iterations for training generative models. Unfortunately, with VAEs and GANs, tracking losses is indeed cumbersome and not necessarily worthwhile.</p>
  <p>If you wish to recreate the results of a paper, your best guess is to train for the same number of iterations and with the same settings.</p>
  <p>If you wish to use this model for a project, then training it until completion is your best bet. However, if your only goal was to understand the paper and be able to implement it yourself, then I personally wouldn't bother too much about exactly recreating the results, and just move on to the next challenge.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-19T14:10:57.427Z
Answers.Text: Yes, my goal is "to understand the paper and be able to implement it yourself". But if I can't repeat the results of the paper at least in some form, will I be successful with more complex models, papers and architectures?
Answers.UserDisplayName: Тима 
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-19T13:49:19.130Z
Answers.LastActivityDate: 2024-06-19T13:49:19.130Z
Answers.UserDisplayName: Robin van Hoorn
Body: <p>When we are training a really DEEP and complicated network (CYCLEGAN, VQ-GAN, VQ-VAE2), how to estimate required time (to be accurate &quot;training steps&quot;) for training process?
  Because from the loss it is not always clear, that the model is in the minima or it is just a part of the training process?</p>
  <p>example: i am training the vq-vae-2 model with the same architechure (i guess) as in the <a href="https://arxiv.org/abs/1906.00446" rel="nofollow noreferrer">paper</a>
  I have trained this model for 40 hours, which is quite a lot (~250 000 steps), but results are not impressive. <a href="https://i.sstatic.net/piRBdJfg.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/piRBdJfg.png" alt="enter image description here" /></a>
  <a href="https://i.sstatic.net/9QsYhaXK.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9QsYhaXK.png" alt="enter image description here" /></a>
  Should i stop the training with the failure to repeat the results of the paper or should i train more (in the paper they trained it for 10x more steps than me)?</p>
  <p>PS: these are results of my previous model (vq-vae-1) with the same number of steps(~250 000)
  <a href="https://i.sstatic.net/Qs4TPPbn.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Qs4TPPbn.png" alt="enter image description here" /></a></p>
  
Tags: <generative-adversarial-networks><generative-model><image-generation>
Title: How long do i need to train a really DEEP network?
UserDisplayName: Тима 
ThreadText: # How long do i need to train a really DEEP network?
  
  (Asked by: username_0 on 2024-06-19)
  
  When we are training a really DEEP and complicated network (CYCLEGAN, VQ-GAN, VQ-VAE2), how to estimate required time (to be accurate "training steps") for training process?
  Because from the loss it is not always clear, that the model is in the minima or it is just a part of the training process?
  
  example: i am training the vq-vae-2 model with the same architechure (i guess) as in the [paper](https://arxiv.org/abs/1906.00446)
  I have trained this model for 40 hours, which is quite a lot (~250 000 steps), but results are not impressive. [![enter image description here](https://i.sstatic.net/piRBdJfg.png)](https://i.sstatic.net/piRBdJfg.png)
  [![enter image description here](https://i.sstatic.net/9QsYhaXK.png)](https://i.sstatic.net/9QsYhaXK.png)
  Should i stop the training with the failure to repeat the results of the paper or should i train more (in the paper they trained it for 10x more steps than me)?
  
  PS: these are results of my previous model (vq-vae-1) with the same number of steps(~250 000)
  [![enter image description here](https://i.sstatic.net/Qs4TPPbn.png)](https://i.sstatic.net/Qs4TPPbn.png)
  
  **Tags:** generative-adversarial-networks, generative-model, image-generation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-06-19. Score: 0
  
  There is no golden rule on number of iterations for training generative models. Unfortunately, with VAEs and GANs, tracking losses is indeed cumbersome and not necessarily worthwhile.
  
  If you wish to recreate the results of a paper, your best guess is to train for the same number of iterations and with the same settings.
  
  If you wish to use this model for a project, then training it until completion is your best bet. However, if your only goal was to understand the paper and be able to implement it yourself, then I personally wouldn't bother too much about exactly recreating the results, and just move on to the next challenge.
  
  > **Comments:**
  > **username_0**: Yes, my goal is "to understand the paper and be able to implement it yourself". But if I can't repeat the results of the paper at least in some form, will I be successful with more complex models, papers and architectures?  
  
  ---
  
  