Hi, can you try first setting the environment variable  TORCH_DISTRIBUTED_DEBUG=INFO. If you need more detail, you can set it to  TORCH_DISTRIBUTED_DEBUG=DETAIL. You will need to use nightly builds with this environment variable set.