Could you rerun your code via CUDA_LAUNCH_BLOCKING=1 python script.py args and post the stack trace here?
The error points to a wrong kernel launch config. Are you using any custom CUDA code in your application? If not, which PyTorch and CUDA version as well as GPU are you using?