If you are using both methods as the forward pass, I’m wondering, how it is working in any approach at all.
In both methods you are reinitializing all layers from scratch, so that no real training will happen.