Machine Learning, Mpi, Pytorch, TensorFlow.
locally, but it’s slower than Ray on network because it relies on single TCP connection. TensorFlow uses gRPC to send information and the decoding seems inefficient — it is bottlenecked by CPU. Breaking up 100MB tensor into smaller chunks makes it closer to PyTorch performance over network.