Matthew_Chen37:

This was all done in Jupyter Notebook by the way.


Setting CUDA_LAUNCH_BLOCKING=1 after the CUDA context was already created won’t have any effect, which seems to be the case here. Thus I do not recommend to set this env variable inside the script and be especially careful when running in Jupyter notebooks (restart the Kernel after each failure to avoid confusing sticky failures).
Use CUDA_LAUNCH_BLOCKING=1 python script.py args (or export the env variable) and rerun your code.