<p>That depends on the value of your loss.</p>
<p>If it is still too high, your model might be stuck in a local minimum. In this case early stopping and another random weight initialisation will help.</p>
<p>You might also need to change other hyper parameters. Another reason could be that your learning rate is too high and you overshoot the minimum.</p>
<p>If it is low enough and there has been no change for 3-5 epochs, it could be the global minimum. In that case, stop early.</p>
<p>Edit:
The possibility of an increase in human-observable image quality while the loss stays the same depends on the metric of your loss. Most likely no because usually the evaluation metric of the loss will cover that.</p>

CC BY-SA 4.0
2022-05-24T18:41:45.517Z
Thanks, but my question is somewhat different. What I meant was the possibility of an increase in human-observable image quality while the loss stays the same.
Roman
CC BY-SA 4.0
2022-05-25T02:56:34.833Z
That depends on the metric of your loss but most likely no because usually the evaluation metric of the loss will cover that.
emely_pi
CC BY-SA 4.0
2022-05-26T00:09:54.360Z
Sounds good, if you edit your answer accordingly. I'll accept.
Roman
CC BY-SA 4.0
2022-05-23T13:33:14.050Z
2022-05-26T00:35:54.983Z
2022-05-26T00:35:54.983Z
emely_pi
<p>I am training a StyleGAN-3 using one of the pre-trained models. At some point, roughly halfway through the 5000 kimg recommended for fine-tuning, the FID50K score starts oscillating around a minimum value and appears to make no further progress. From your experience, can the model still make further improvements to generated images that will be noticeable to the human eye but not reflected in the FID50K, or should I stop training early?</p>

<training><generative-adversarial-networks><early-stopping>
Training a GAN after after evaluation metric reaches minimum
Roman
# Training a GAN after after evaluation metric reaches minimum

(Asked by: username_0 on 2022-05-12)

I am training a StyleGAN-3 using one of the pre-trained models. At some point, roughly halfway through the 5000 kimg recommended for fine-tuning, the FID50K score starts oscillating around a minimum value and appears to make no further progress. From your experience, can the model still make further improvements to generated images that will be noticeable to the human eye but not reflected in the FID50K, or should I stop training early?

**Tags:** training, generative-adversarial-networks, early-stopping


---

## 1 answers

### Answer by username_1 on 2022-05-23. Score: 0 | Accepted answer

That depends on the value of your loss.

If it is still too high, your model might be stuck in a local minimum. In this case early stopping and another random weight initialisation will help.

You might also need to change other hyper parameters. Another reason could be that your learning rate is too high and you overshoot the minimum.

If it is low enough and there has been no change for 3-5 epochs, it could be the global minimum. In that case, stop early.

Edit:
The possibility of an increase in human-observable image quality while the loss stays the same depends on the metric of your loss. Most likely no because usually the evaluation metric of the loss will cover that.

> **Comments:**
> **username_0**: Thanks, but my question is somewhat different. What I meant was the possibility of an increase in human-observable image quality while the loss stays the same.  
> **username_1**: That depends on the metric of your loss but most likely no because usually the evaluation metric of the loss will cover that.  
> **username_0**: Sounds good, if you edit your answer accordingly. I'll accept.  

---

