Hi, I wasn’t able to repro your issue with torch-1.10.
Here’s the test script I tested with
# save this as test.py

import argparse
import os
import sys

from torch.distributed.elastic.multiprocessing.errors import record


def parse_args(argv):
    parser = argparse.ArgumentParser(description="test script")
    parser.add_argument("--local_rank", type=int)
    return parser.parse_args()

@record
def main():
    args = parse_args(sys.argv[1:])
    print(f"local_rank={args.local_rank}")

if __name__ == "__main__":
    main()

Then run
$ python -m torch.distributed.launch --nnode=1 --node_rank=0 --nproc_per_node=4 test.py
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  warnings.warn(
WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
local_rank=3
local_rank=2
local_rank=0
local_rank=1

The example seems out of date. Please follow the instructions here: torchrun (Elastic Launch) — PyTorch 1.10.0 documentation 5