Hi,
Can you run your script with  CUDA_LAUNCH_BLOCKING=1 python your_script.py  to get a more accurate stack trace?
Besides can you give us more information about your problem? likes the labels you use?