❓ [BART] Why Decoder Layer Normalization is applied only at the last layer ? 
# ❓ Questions & Help

It seems this line :
https://github.com/huggingface/transformers/blob/efbc1c5a9d96048ab11f8d746fe51107cb91646f/src/transformers/modeling_bart.py#L524

was modified when MBART was added. 

---

Before, Layer Normalization was applied after **all** layers of the decoder (_similar to the encoder, if the config was set appropriately_).

But now, Layer Normalization is applied **only at the end**, even for other BART models (_not MBART_).

---

**Is it expected ? What's the reason behind this logic ?**

@sshleifer 
That `layer_norm` should be None in `bart-large*`.
See [this comment](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bart.py#L266)

No final `layer_norm` was applied before `mbart` was added, afaict.