Solved by ptrblck in post #2 
              
              
                You could use streams, but would have to make sure to properly synchronize the code to avoid race conditions. 
Depending on the actual workload on the device, your speedup might not be huge, e.g. if the first model already uses the device sufficiently.