Oh; I have finally found out why it’s working now !!! It’s because the way I compute loss some how involves some globally shared parameters that requires retain_graph=True !  Thanks so much for all tips