I implemented the Checkpointing as you said and, after doing some tests, I saw a little (really little) decrease in GPU memory (for this purpose I used some funcions from torch.cuda).
When I started to implement the Checkpointing I thought I would obtain a more signficant differences on GPU memory usage, I suposse that my architecture does not fit well in this framework.
Thank you for the help!