I didn’t set it every iteration. I only set it once. I have added one line in the loop after forward the input
            print(t,syn_energy.sum(),sample_seq.requires_grad)

The ouput is as follows:
0 tensor(-1.0209, device=‘cuda:1’, grad_fn=SumBackward0) True
1 tensor(6.3761, device=‘cuda:1’, grad_fn=SumBackward0) True
However, sample_seq.grad is None.