Great!
For merging logs, I see a few options. There might be more.

Let one rank do the printing (e.g., if rank == 0: ...)
If you need logs from all ranks, you can redirect then to write to different files.
Convert strings to tensors, and then use all_gather/gather to collect them (https://pytorch.org/docs/stable/distributed.html#torch.distributed.gather 13). This might be a little more complex than it sounds. This 10 can serve as an example.