Hey,  do you find the root cause of this issue?
I have a similar problem after updating pytorch from 3.0 to 4.1.
The training becomes unstable using the p100.
Would you recall the reason for this?
thanks