This does not work. If I do that I don’t get any gradients. I think you can not backprop through a graph if part of it was calculated inside torch.no_grad(),i.e. the network is blind to anything that happens after the no_grad() section. I think this does not work for the same reason than you do not get gradients if you write your loss function inside torch.no_grad().