One option is to use nvprof 1.

In my training the results of single node and distributed training appear similar.

You mean speed is similar? What is the batch size fed into each DDP instance? When using DDP, the batch_size should be updated to original_batch_size/world_size.