sunshichen:

Yes. Validate on all GPUs works. But shouldn’t model.eval() aotomatically disable sync process?


Good point, I’ve opened an issue for this: https://github.com/pytorch/pytorch/issues/48988 26