Thanks for the links. ELF is not written in pytorch and is quite heavy-weight. It’d be much more illuminating to see minimal example code with torch.distributed to reproduce at least part of the TF parameter server + async training logic. Furthermore, the async GPU skeleton code can be reused over and over again in many cases.