Hi, I am not sure whether you have solved the problem, but I can share you my experience on this headache error.
I just meet the same runtime error suddenly when I run my previous bug-free codes to train the network with 2 GPUs simultaneously. The error is thrown out after several epochs and it even makes my machine freeze totally. However, the training script is exactly the same and it has worked for some weeks before. After I re-configure the environment, the problem still exists. Finally, when I test my script with single GPU card one by one, I realize one of my GPU cards is broken.  So I recommend you do some GPU pressure test to check the GPU itself. Hope this late reply be useful to other guys suffering the same bug.