Hey @matrix, sorry about the delay again. We really need to work on our oncall procedure to cover pending discussions not just new discussions.
For the timeout error, the first thing I would check if whether the network is indeed working. To do that, one way is to call rpc_sync (not remote or rpc_async) between all pairs of nodes. If that works, it means the network indeed works. From the timeout message, I cannot tell why the distributed autograd does not work. If you could share your latest code with me, I can grab three machines on AWS and try it.