Thanks for the response; yes, you’re totally right. After comparing the initial state of the parameters of the two models, it turned out that they differed in every nn.Linear() weight initialization but had the same weight and bias initializations for the BatchNorms (nn.BatchNorm1d()).
I guess the question now is how to enforce the same weight initialization for all the nn.Linear() modules in both models?