Hello,
My research is about distributed deep learning, and I am looking for a research developing platform in which new ideas could be implemented. Regarding complex architecture, we should have more access to communications between nodes and GPUs.
I found TF-Replicator from Tensorflow, I saw Pytorch RPC and DDP APIs, and also Ray that is a new designed intesteing platform for emerged AI.
I could found any example and tutorial (Good ones) for TF-Repliicato or Ray. However, Pytorch has some good documents and I feel it gives more control over how we implement more complicated architecture and communication schema.
Therefore It would be great if others can also share their thoughts.
Thank you