Ok, I got this to work, because apparently the indexing of the GPUs used by nvidia-smi, is not the same as the indexing used by the program. This was what was causing on of the issues. (There was another issue where if the batch size was too small, it complained, but that seemed to go away once I made the batch size larger. Thanks!!