temporally disable .requires_grad works! I have not found a better solution than that so far. I am detaching z in distance(z_hat,z.detach()), because I don’t want the encoder to adapt explicitly to the “bad” encoding z_hat.