Hi, thanks for your reply! I’m still a bit confused about the concept rank. Does it correspond to the number of gpus available?
For example, if I’m using 1 node with 4 gpus to train, how should I set those parameters?