Hi,
In the error output, if you look closely, you will see a recommendation to gather more information about the root cause:



 adelaide:

CHILD PROCESS FAILED WITH NO ERROR_FILE
Child process 24711 (local_rank 0) FAILED (exitcode 1)
Error msg: Process failed with exitcode 1
Without writing an error file to <N/A>.
While this DOES NOT affect the correctness of your application,
no trace information about the error will be available for inspection.
Consider decorating your top level entrypoint function with
torch.distributed.elastic.multiprocessing.errors.record. Example:
from torch.distributed.elastic.multiprocessing.errors import record
@record
def trainer_main(args):


You can find more information about the record decorator here 26. If you decorate your main function with @record you should get a full stack trace of the actual exception.
Having said that I would also suggest running your code with the --max_restarts=0 option. This was a bug in v1.9 that is fixed in v1.9.1. Without that option, your training will be restarted up to 3 times before ultimately failing.