hi, I got another irreproducibility problem.
I downgraded the RTX2080TI platform from torch v1.0.0 to torch v0.4.0, and now it is reproducible when running the code >= 2 times.
But I found that the batchnorm and nn.linear cause irreproducibility between my two platform below.
If I remove the batchnorm and replace the nn.linear by nn.conv2d, it is reproducible.

torch v0.4.0, Cuda 8.0, Cudnn 6.0.21, Ubuntu 16.04, GPU GTX1070.
torch v0.4.0, Cuda 9.0, Cudnn 7.4.2.24, Ubuntu 18.04, GPU RTX2080TI

Actually, both CPU-version and GPU-version are irreproducible.