I’m not sure what your code is actually doing, but I get valid gradients even inside forward:
...
adv_distance.backward()
print(d.grad)
print(self.d_dummy.grad)
...
> tensor([[ 1.7061, -0.8534,  0.1437,  0.7026,  0.3845,  0.9892,  1.0329, -1.8058,
         -2.0117, -0.1880]])
tensor([[ 1.7061, -0.8534,  0.1437,  0.7026,  0.3845,  0.9892,  1.0329, -1.8058,
         -2.0117, -0.1880]])

It looks like you are trying to create some adversarial sample inside the forward of VATLoss, but apparently you are recreating it in each iteration from scratch?
I think it might be better to create a new topic and explain your use case a bit more so that others might have a look at this issue.