Thank you for making me discover checkpoints ! They are very useful to trade compute for memory.
I’m afraid that since i’m checkpointing all of the pre-trained model, the tensors are not requiring grad anymore.
I get this error :
  UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")

I made the input requires grad. I hope this will solve the problem.
Thank you