Small error in documentation / Typo
The documentation for BART decoder layer mentions that it expects hidden states as well as the encoder hidden states to be expected in "(seq_len, batch, embed_dim)" instead of the "(batch, seq_len, embed_dim)" that is actually is expected. Lead to a bit of confusion so would be great if it were corrected! :)

Relevant lines:
https://github.com/huggingface/transformers/blob/996a315e76f6c972c854990e6114226a91bc0a90/src/transformers/models/bart/modeling_bart.py#L373
https://github.com/huggingface/transformers/blob/996a315e76f6c972c854990e6114226a91bc0a90/src/transformers/models/bart/modeling_bart.py#L376

@patrickvonplaten 
Thanks for spotting. Feel free to open a PR to fix this :) 

By the way, I see you're the main author of MDETR (amazing work!). I'm currently adding DETR to the repo (see #11653), so if you are up to help me add MDETR to the repo, feel free to reach out :) 
<|||||>Ooh thanks :D That sounds great, I'd be happy to help :) Will send you an email. 