To be honest: I don’t know.
Your error looks pretty strange, but since it might be related to your data it can’t be wrong to have a look on your data loading.
Do you set your CUDA_VISIBLE_DEVICES before or after importing torch or modules, that use torch?