Thankyou ptrblck, I get this
THCudaCheck FAIL file=../torch/csrc/cuda/Module.cpp line=37 error=101 : invalid device ordinal

Just share here in case others encounter the same error.
If set
CUDA_VISIBLE_DEVICES =3

then in the train.py code, the gpu_id should be set to 0, not to 3.