for noobs like me:
use
import os
os.environ["CUDA_LAUNCH_BLOCKING"] = "1"

on top of your script and run the script to get a way better stack trace. in my case  the cuda error told me to look at a completely different line which caused an hour of debugging