Solved by albanD in post #2 
              
              
                Hi, 
When you do checkpointing, the forward is called twice. 
Once during the forward without tracking gradients (and so requires_grad=False) and once, during the backward, with tracking enabled to recover all the buffers.