@zjoe I just pushed a patch that improves DataChannelMPI in a way that should make it CUDA compatible and improve performance. Can you let me know what are the numbers if you build from this branch? https://github.com/pytorch/pytorch/pull/3817 25
Thanks!