XW324:

An obvious way is to apply models in a sequential way inside the same dataloader iteration, but would it make use of my gpu efficiently?


No, you’d only amortize data loading time. May be worth it if it is a notable proportion of iteration time (data loading+forward+backward).



 XW324:

My naive guess is that if multiple models can be run in a parallel fashion under inside the same dataloader iteration then that would fully make use my single GPU.


Yes, but they won’t run forward() in parallel, unless you write code for it. Here, first problem is you would need num_models times more gpu memory (probably more due to increased fragmentation); another complications include python GIL and need for cuda streams.