Nevermind, a stupid mistake on my part ^^’
I was doing:
from torch.utils import checkpoint

instead of:
from torch.utils.checkpoint import checkpoint

It was a good try, but it only saved 5Mb in my case.
I still appreciate it, thanks!