Thanks @Kiuk_Chung for your answers. I will check the etcd FAQ.
I’m thinking of a parameter server deployment with a crash tolerance to the central server. I want to deploy, let’s say, 2 servers so that if one crashes, the other one takes over. Yet, I want this to be transparent to the workers, i.e., they still send their gradients normally to the process with rank 0.
Is there an easy and cheap way to do this currently?
Thanks!