OOM when using DDP would lead to de-synchronization across processes in the same group, which in turn would cause hang or crash. TorchElastic is built to solve this problem. When there is an error, it will destroy all DDP instances on all processes, and then re-construct a  new gang. Is this sufficient for your use case?