@ngimel, Thank you for the responses.
I’ve set CUDA_VISIBLE_DEVICES=0 (I have a single card installed at the moment) and no change.
Ran the original script main.py 6 as well as main_fake.py - yield the same results.
Reduces the transform to merely ToTensor call as well. I don’t think it’s related to the data loader.
I do see that my batch-size limit in terms of memory is ~212 (could it be that the reason for that is that the card is used for display as well hence less memory available ?)
Another thing to note is that while gpu utilization shown in nvidia-smi is high (~95%) the power usage is ~120W / 250W. Is that normal ?