I’m not sure what the exact issue is, can you post a fuller example to run?
Alternatively read our new distributed tutorial here: http://pytorch.org/tutorials/intermediate/dist_tuto.html 191 to see if it helps.