Regarding this, I did two experiments.


On my own desktop, using the “bad” GPU and another GPU.
On the “bad” GPU, the two example scripts report errors (when autograd anomaly detection is activated). On the other GPU, no error is reported.


On another desktop, the same errors can be reproduced with the “bad” GPU. Using another GPU on this another desktop will not report errors.


The “bad” GPU is RTX2080Ti, while the other GPU is 1080Ti (because no available 2080Ti at hand). I don’t think it’s PyTorch’s problem; it shall be a hardware issue.
I don’t know if this is a too-much request for PyTorch: Is it possible to have a small testing code in PyTorch (or there has been one but I am just not aware) to check if a GPU is faulty? In this way, it might be helpful to identify if this is a hardware or software problem. Anyway, it might be too much for PyTorch, then just forget it.