Solved by ptrblck in post #2 
              
              
                Could you update to the latest PyTorch release and also try the CUDA10.2 or 11.0 binaries? 
If you are still hitting the illegal memory access, could you run the script via: 
CUDA_LAUNCH_BLOCKING=1 python script.py args

and post the stack trace here?