I’m not sure. If you call torch.load it should restore the tensors on the device prior to saving.
Optimizer 6 also casts the tensors to the respective device.
I can’t really tell without the complete code.