ptrblck:

In case of checkpointing,


Very cool!
Until now I tried doing requires_grad_() on the actual input (which effects the computation and creates overhead) - but the idea of additional dummy input which isn’t used in the computation sounds great!!
Trying it now and will update.