Hi,
in torch.distributed:
node means the machine(computer) id in the network.
rank, means global_rank, means the process id in the network
local_rank, means the process id in local machine(computer).
Is the my understanding above correct?
in my code,
local_rank could be read from args.local_rank if torch.distributed is applied. otherwise local_rank doesn’t exist in args, right?
how could I get node id and rank in my code?