As all processes are independent from each other without communications, self.x should be explicitly handled for all processes with different ranks to have the same value.
You can refer to the collective functions in pytorch distributed docs.
You may choose to broadcast or all_reduce, etc.