Hi, I am trying to use DistribuedDataDarallel for multi-node data-parallelism.
I want to know how can I measure the time breakdown for data load, forward, backward, communication?
Also, for calculating FLops, I am going to use the repository[Calculating flops of a given pytorch model 4] in github. Does anyone know the good way for calculating Flops.