Could you run your code while setting CUDA_LAUNCH_BLOCKING=1 python your_code.py and see where the error comes from please?
The cuda api is asynchronous by default so the stack trace above might we wrong.