Try to run the code using
CONDA_LAUNCH_BLOCKING=1 python script.py args

so that all CUDA operations will be synchronized. The error message should point you to the right line of code which is throwing the error. Once you found the line, check the device of the data using print(data.device) and make sure to push them to the device.
Also, if you are using vanilla Python lists to store some submodules, use nn.ModuleList instead, since Python lists won’t be pushed to the device by default.