Solved by ptrblck in post #2 
              
              
                Most likely you won’t see a performance benefit, as a single ResNet might already use all GPU resources, so that an overlapping execution wouldn’t be possible. 
If you want to try it nevertheless, you could have a look at the CUDA streams docs. Note however, that you would be responsible for proper …