In this case, it would not make much difference because- the number of layers for both the encoder and decoder is the same. However, if the number of layers is unequal then the choice of which layers (the first decoder.n_layers or the last decoder.n_layers) to use is purely dictated by personal design choice.