Your data loader looks OK, do you suspect that it is slow on loading the data?
Increasing batch size might increase the GPU utilization, but it also affects the learning process, mini-batches have an important part in training, they provide generalization (some would argue that batch_size=1 will be the best 2 but no need to go too extreme)
So keep an eye on the accuracy 
Roy