You can use it in case you are seeing invalid outputs or gradients, so could this be an issue for your use case?
I mentioned mixed-precision training (torch.cuda.amp), since gradients might overflow and the GradScaler would skip these updates. These iterations might however trigger the anomaly detection, so you should activate anomaly detection once the loss scaling factor is stable.
If you are not using torch.cuda.amp, then something might still be broken in your code and I would like to debug it in this case.