Thank you so much!
On MPI, I can use distributed object to remote other workers do something with other worker’s distributed object.
On pytorch, I saw that it has Remote Reference (RRef). However, I did not see how to create distributed object with Pytorch. Could you please suggest in this case?
For example,
worker1 holds a dictionary with key and value DIC1.
worker0 will send an array KEY_SENDs with the keys to worker1. Worker1 will check DIC1 and return an array with the values from KEY_SENDs and DIC1.
I try with
def get_values(dic1, arr):
    rest =  np.array([])
    for i in arr:
        rest= np.append(rest,dic1[i])
    return rest
        

def run(rank, size):
    tensor = torch.zeros(1)
    my_name = "worker"+str(rank)
    if rank == 0:
        thisdict =	{4:2,2:6,3:8}
        rpc.init_rpc(my_name,rank=0, world_size=size)
        target = 1
        target_name = "worker"+str(target)
        ret = rpc.rpc_sync(target_name, get_values, args=(thisdict,[1,2]))
        print(str(ret) + " is in rank 0 from rank 1")
    else:
        thisdict =	{2:1,1:3,4:2}
        rpc.init_rpc(my_name,rank=rank, world_size=size)
    rpc.shutdown()
    print(my_name)


I run it but no results were show on this example.
Thanks,