Hi, sorry for the late reply I am currently very busy.
I decided to use the Pytorch c10d library instead, thus I can’t tell you how to do it with the RPC package. I found it quite intuitive to use the p2p and collective functions of c10d. The collective functions also allow you to combine gradients very easily (by summing).
Please let me know in case you need any further details about the c10d library and my implementation.
I hope this doesn’t come to late…