Is there a way to enable distributed inference, instead of training? Also, is it possible to distribute the work across multiple servers each with multiple GPUs, or does it only work for a single server with multiple GPU? If any of these features are missing, will they be coming out soon?
Lastly, what would be the recommended environment / library to enable distributed inference on multiple servers each with multiple GPUs?
Thanks!