If your optimizer contains internal states (running estimates of the gradients etc.), it would probably make sense to restore it. Otherwise you might see a peak in your loss when you continue the training on the new dataset. I don’t think the loading of the optimizer is dataset-dependent, but let us know, if you see a different behavior for your use case (which would be interesting to know).