I was also having the same problem, but for me it was:
with torch.autograd.set_detect_anomaly(True):
    train(hyperparameters)

I did that to check if my gradients are exploding or not, then I forgot to remove that context manager.
It would be helpful if someone can explain why this was happening. Thanks