Answers.Body: <p>That depends on the value of your loss.</p>
  <p>If it is still too high, your model might be stuck in a local minimum. In this case early stopping and another random weight initialisation will help.</p>
  <p>You might also need to change other hyper parameters. Another reason could be that your learning rate is too high and you overshoot the minimum.</p>
  <p>If it is low enough and there has been no change for 3-5 epochs, it could be the global minimum. In that case, stop early.</p>
  <p>Edit:
  The possibility of an increase in human-observable image quality while the loss stays the same depends on the metric of your loss. Most likely no because usually the evaluation metric of the loss will cover that.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-24T18:41:45.517Z
Answers.Text: Thanks, but my question is somewhat different. What I meant was the possibility of an increase in human-observable image quality while the loss stays the same.
Answers.UserDisplayName: Roman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-25T02:56:34.833Z
Answers.Text: That depends on the metric of your loss but most likely no because usually the evaluation metric of the loss will cover that.
Answers.UserDisplayName: emely_pi
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-26T00:09:54.360Z
Answers.Text: Sounds good, if you edit your answer accordingly. I'll accept.
Answers.UserDisplayName: Roman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-23T13:33:14.050Z
Answers.LastActivityDate: 2022-05-26T00:35:54.983Z
Answers.LastEditDate: 2022-05-26T00:35:54.983Z
Answers.UserDisplayName: emely_pi
Body: <p>I am training a StyleGAN-3 using one of the pre-trained models. At some point, roughly halfway through the 5000 kimg recommended for fine-tuning, the FID50K score starts oscillating around a minimum value and appears to make no further progress. From your experience, can the model still make further improvements to generated images that will be noticeable to the human eye but not reflected in the FID50K, or should I stop training early?</p>
  
Tags: <training><generative-adversarial-networks><early-stopping>
Title: Training a GAN after after evaluation metric reaches minimum
UserDisplayName: Roman
ThreadText: # Training a GAN after after evaluation metric reaches minimum
  
  (Asked by: username_0 on 2022-05-12)
  
  I am training a StyleGAN-3 using one of the pre-trained models. At some point, roughly halfway through the 5000 kimg recommended for fine-tuning, the FID50K score starts oscillating around a minimum value and appears to make no further progress. From your experience, can the model still make further improvements to generated images that will be noticeable to the human eye but not reflected in the FID50K, or should I stop training early?
  
  **Tags:** training, generative-adversarial-networks, early-stopping
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-05-23. Score: 0 | Accepted answer
  
  That depends on the value of your loss.
  
  If it is still too high, your model might be stuck in a local minimum. In this case early stopping and another random weight initialisation will help.
  
  You might also need to change other hyper parameters. Another reason could be that your learning rate is too high and you overshoot the minimum.
  
  If it is low enough and there has been no change for 3-5 epochs, it could be the global minimum. In that case, stop early.
  
  Edit:
  The possibility of an increase in human-observable image quality while the loss stays the same depends on the metric of your loss. Most likely no because usually the evaluation metric of the loss will cover that.
  
  > **Comments:**
  > **username_0**: Thanks, but my question is somewhat different. What I meant was the possibility of an increase in human-observable image quality while the loss stays the same.  
  > **username_1**: That depends on the metric of your loss but most likely no because usually the evaluation metric of the loss will cover that.  
  > **username_0**: Sounds good, if you edit your answer accordingly. I'll accept.  
  
  ---
  
  