Ah ! Making progress 
So with the Quadro GP100 and a source install I can’t reproduce that.
Do you see the same behavior on nightly build for this test script?
Which version of cuda are you using and how did you installed pytorch?