Your workflow can launch the model execution on different devices and they will be executed asynchronously unless you are synchronizing explicitly (via torch.cuda.synchronize()) ir implicitly e.g. via tensor.nonzero(), tensor.item() etc.
If you are not seeing overlapping execution in e.g. Nsight Systems, your CPU (and the overall dispatching) might be too slow compared to the GPU execution time.