Sandro_Giacomuzzi:

i.e. the network is blind to anything that happens after the no_grad() section


But your z_hat = E(x_hat) is the final segment, so why is this a problem? To apply a loss like distance(z_hat, z), z_hat gradient is not needed.