@mrshenli

Can you please define the terms: 1) node, 2) process in the context you are using them?
If I want to train my model on 4 GPUs, do you call it 4 processes? or 1 process?
Does init_method correspond to the address of my PC or to the GPU I’m accessing on a cluster?
In this 1 tutorial, what were you referring to as machine?

@mingyang94

Can you please explain how you arrived at:




 mingyang94:

init_method