You can use non-blocking data transfers using the non_blocking=True argument in e.g. tensor = tensor.to(). This NVIDIA blog post 170 gives you some information what’s going on under the hood.
In case I misunderstood your question, could you clarify it a bit?