I am not sure I understand why there’s a reference cycle created. Could you please elaborate on this?
Also, I called self.zero_grad() after the backward operation. If there’s any reference cycle created, they are supposed to be removed by zero_grad(), right?