Thanks for the reply @ptrblck : )
I do need gradients calculated, and the reason that I’m using it is to trade memory and performance speed.
I probably did not explain myself well, sorry.
See, for example, the minimalist reproducing code that was mentioned in the github issue that I linked above:
import torch
from torch.utils.checkpoint import checkpoint

m = torch.nn.Linear(4,3)
x = torch.randn(10,4)

z1 = checkpoint(lambda _:m(_), x)
print(z1.requires_grad)

z2 = m(x)
print(z2.requires_grad)

If you run this code it will output:
.../miniconda3/envs/py36torch/lib/python3.6/site-packages/torch/utils/checkpoint.py:20: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
False
True

As you can see, no gradients will be computed in the checkpoint example (for all of the parameters inside torch.nn.Linear(…) ).
And on the other hand, gradients WILL be computed in the example that doesn’t use checkpoint. (Even though, all of its input have requires_grad=-False    ! )
This is most problematic when you want to use checkpoint as the first operation(s) in your network.
For example, to explain my motivation - I want to use it to calculate features for several images and then combine the features (and have all in one computation graph).
Each image feature extraction is checkpointed, so hopefully this will allow me to not get OOM.
Of course, maybe I’m missing something here, so please enlighten me if I do : )