@ ptrblck, Thanks for your response.
It is true that we can get the exactly same outputs if transferring the state of one model to the other model. However, my real confusion is specifically for my example, why the order of the layer initializations can lead to different weights/bias values. My understanding is there shouldn’t be any dependencies between the parameter initialization of any two different layers. For example, Conv2 initialization doesn’t depend on any outputs from Conv1. Any thoughts?
Best
GR