Hi,
If I understand correctly, the intermediate activations will be stored for the layers which we checkpoint and for the others will be recalculated at the backward pass. So basically we need to checkpoint the block which we want the activations to be stored and not the block for which we dont want to store. Is my understanding correct?
Regards