Thank you, @ayalaa2 ! I understand that if I used all the ranks for validation it would be faster. I will try to share tensor values across ranks.



 ayalaa2:

ssuming that you’re also only logging with rank 0 as well?


Yes, I am using wandb only with rank 0.