How large is your batch size?
Also, the statement CUDA_VISIBLE_DEVICEs=1,3,4 won’t have any effect.
If you would like to set it in your script, you should do it before importing torch or any other library which initializes CUDA and also use os.environ['CUDA_VISIBLE_DEVICES']=1,3,4.
However, I would recommend to set it as an env variable in your terminal by executing your script via:
CUDA_VISIBLE_DEVICES=1,3,4 python script.py args