Solved by mrshenli in post #2 
              
              
                Hey @karthi0804, could you please share code of a repro? 
This error is from here, indicating that parameter sizes/order do not match across processes. Could you please verify if that is the case by printing sizes of params in model.parameters()? 
cc @Yanli_Zhao