I am setting different seed for each process (distributed training setting) but the models weights are still same for all the processes. Is it an expected behavior?
PS: I was under the impression that since weight init depends on seed so, different seed will lead to different weights init for each process.