Machine Learning, Mpi, Pytorch, TensorFlow.
— run SGD independently on multiple machines and periodically average their parameters. Example above uses “resource” tag to control where your computation actually runs. I’m using machine=1 pattern to ensure that only one actor gets instantiated per machine. For peak performance you’d probably use