class MainModule(nn.Module):
...
self.proj = Mlp()
<end MainModule>


output = self.main_module(output)

class MainModule(nn.Module):
...
<end MainModule>

output = self.main_module(output)
output = self.proj(output)

These two bits of code should be exactly the same, yet they consistently produce very different results for every run. How can I debug this?
If I somehow can ensure that self.proj has the exact same weights (I don’t think setting the random seed works when the code itself is different from run to run) will let me at least see if the forward activations and gradient updates are different… how do I do that?