Hi,
When you do checkpointing, the forward is called twice.
Once during the forward without tracking gradients (and so requires_grad=False) and once, during the backward, with tracking enabled to recover all the buffers.