I have a very similar error. The line causing it is optimizer.step(), but it only seems to happen after loading from a checkpoint. I am sure to move the model to the proper device after loading, is there something else we need to call .to on?