I’m not sure, but my guess would be the internal functionality of RMSProp.
This optimizer divides the gradient by a running average of its recent magnitude.
If your gradients are quite small, since you’ve already trained your model for a few epochs, I assume this division might blow up.