I’m not deeply familiar with higher, but it seems that the DifferentiableOptimizer base class accepts an already created optimizer object, so I would assume that you could still query the original optimizer.state_dict()? Could this be the case or are the original states stale?