Machine Learning, Mpi, Pytorch, TensorFlow.
while, and there are others like scatter, gather and all-to-all. References: PyTorch distributed tutorial mpi4py docs TensorFlow C++ collectives API Task-based Examples: Ray, Dask Execute Python function (Task) somewhere else and get its return value lazily. This may be the most natural for Python