Thanks for the prompt reply. If CONV layer does not need the output for its backward pass, I wonder how its gradients are calculated. In addition, how do I know which layer needs the output for its backward pass (so we can have in place update). I didn’t find any related info from Pytorch document,  it would be greatly appreciated if some reference could be shared.