Thanks Dhananjay for your time to reply! Those were my intuitions as well. I tried playing with the num_workers parameters as well, however playing around with it does not seem to change much so I have kept it between 8/16 or 32. But as I said, it did not seem to matter much.
My batch size is already 64, I do not want to make it larger than that in case that makes it harder for the network to generalize (As far as I understand). Is there anything else that can solve this issue for me? Please note that the images are loaded inside the get_item function inside the Dataset class.