Its the later, elastic monitors worker pids and restarts the world whenever one or more pids fail. Elastic was designed to be more general purpose to fit a variety of distributed training use-cases. For pure torch rpc apps it might be possible to allow workers to leave/join the job on the fly, but if you are using process groups the backend may not allow this by design - for instance if you are using NCCL backend, NCCL itself does not allow resizing of communicators unless you destroy them and restart them.