Solved by albanD in post #5 
              
              
                No it’s the oposite. 
By default, the autograd saves all intermediary results needed for the backward pass. 
The checkpointing tool has been added to allow not storing all intermediary results. So you only need it if you don’t have enough memory to store your model, otherwise it will just slow down …