I’m training multiple models using the same datasets. Currently I simply write separate scripts for these models and train them on a single GPU. But as they are using the same dataset, I think my current way of doing things will create a lot overhead on the dataloading part.
So I’m just wondering if there is a way to train multiple models under the same dataloader. An obvious way is to apply models in a sequential way inside the same dataloader iteration, but would it make use of my gpu efficiently? My naive guess is that if multiple models can be run in a parallel fashion under inside the same dataloader iteration then that would fully make use my single GPU.