@sangheonlee Sorry to bother. I wonder which tools you use to profile nccl kernel in this post. I also want to profile multi-gpu training process, but I met some problems. Can you share how you get the profiled timeline? Thanks a lot!