I see. When you create ordereddict, the weights are already initialized for those modules. nn.Sequential is just a container that holds the modules, but it does nothing to initalize the weights.  The final torch.manual_seed(1) is not having any effect on weights in your code.