Solved by ptrblck in post #2 
              
              
                The docs point out that the send is applied synchronously, so the process would wait until the tensor is received. If you want to use the async API use isend and irecv and make sure the tensors are available before using them.