If this is the case, I may have misunderstood how no_grad works. I was under the impression that with no_grad, the gradient would not be computed for the second call of layerA and thus the gradient cannot be propagated to layerB and also the first call of LayerA. Can you confirm this is not the case? Thanks for the quick reply.