I have added a dummy operation that artificially increases the running time when not done asynchronously.
for i in range(3):
            torch.mm(input, input.t())

input array is shape (3000,  50000). Unfortunately the code contains some proprietary stuff so I can’t share, I’ll see if I can make a version I can post here. And I’ll run nvprof