Hi,
Thank you so much!
I search the documents, but I did not know how to make a dictionary as a global value with PyTorch.
I worked with MPI and UPC++. MPI provides a distributed object, which helps each worker to hold their values. UPC++ also provides distributed objects that are a similar name on all workers but different memory locations. Additionally, UPC++ creates Global Pointers to remote from other workers. I quit easy.
However, I did not know these kinds of variables in Pytorch documents.
Do you have any suggestion?
I have another question. If i run with several iterations, is there any function to waiting all workers as barrier() function.
Thanks.