I completely agree with how it’s detrimental to create new optimizers, throwing away history. I’m a bit surprised to not see much slow down/performance hinderance when doing this. It’s definitely important to continue using the same optimizer, but I’m curious to get an idea for how important it is