In the broadcast 1 API, there is an async_op, which by default is False. If it is False, all processes will block until the value is broadcasted. Otherwise, if it is True, broadcast will be non-blocking and return a Future-like object, and you can call wait() on that object. In this case, the tensor is only guaranteed to hold the result after wait() returns.