Ajay-Meena-IITBombay:

I think since patch_embedding_2 is not involved in forward pass it should also not involve in backpropagation


Yes, this should be the case since the module would not influence the calculation of other layers.



 Ajay-Meena-IITBombay:

and the training loss should be identical with/without adding patch_embedding_2 inside init().


That’s not entirely the case, since your unused module could call into the pseudorandom number generator to initialize its parameters, which will thus change the other calls and will not yield the exact same results.
This topic 3 shows some examples and discusses the effect in more detail.