Machine Learning, Mpi, Pytorch, TensorFlow.
multi-GPU implementation optimized for single machine and Ray to parallelize across machines. Resources: Ray documentation Ray team + mine paper: Flexible Primitives for Distributed Deep Learning Implementing Parameter Server with Ray tutorial from Robert Computation graph Examples: TensorFlow