Hi, thanks for the reply.
Regarding DistributedSampler:
What you mentioned is exactly the reason why I raised the question. Since each GPU is fed different data, the accuracy on GPU0 is not necessarily the accuracy on the entire dataset. But as you mentioned, if it is enough to evaluate the model on a single rank, this shouldn’t be an issue.
Regarding AverageMeter:
The  acc_meter.avg should take the average accuracy across all updates, but it actually divides by the batch size again. If you look on L340 1, images.size(0) is passed as a second parameter. On the update() function at L376, the variable n represents the number of values (the count), and the sum is divided by the count to compute the average. Thus, we end up dividing by the batch size twice: once in the accuracy function, and once in the average meter. The second parameter in L340 should be left blank. But again, this is assuming that all batch sizes are same (sometimes the last batch is smaller).
I’ve raised an issue 7 on the repo. I could open a pull request as well. The accuracy issue looks like a minor fix, but adding a DistributedAverageMeter may need some work