Thanks for the update. In that case I don’t know what might cause the wrongly reported time, as the input values shouldn’t get negative (batch size as well as world size) and even if you are not synchronizing the code properly to measure the GPU time, you should not get negative results.
Maybe I’m missing something and someone else has an idea.