Hello,
First, I didn’t find any appropriate category, so I’ll type my question here and I am sorry if it’s not the right place. My problem is the following:
I have two GPUs (TitanXp and 2080 Ti), where I get different results on my training, even after I used all the tips here. I pinpoint the issue in the initialization of the network (using the xavier_uniform_ function) and to be exact after calling Tensor.uniform_. The strange part is that for the whole network (85 modules), a part of each module, first 50 -if I remember correctly- rows with 1024 values are identical and from there until the end of the tensor completely different. Do you have any clues, why that’s happen? Furthermore, do you have any idea/“trick” to get the same initialization on both machines?
Thanks in advance.
George