Thank you. But that would mean that memory is still being used for the later layers during the forward pass right ? I am training a model that uses the output of resnet’s 2nd block and would not want to waste memory.