[fsmt] Expanding Positional Embeddings
splitting the comment from @sshleifer in https://github.com/huggingface/transformers/pull/7224#pullrequestreview-492155076 into this separate issue, as the PR hasn't introduced this potential issue. He writes:

> The expansion of embeddings may require a bit more care, but the comment below doesn't prevent merging this PR. You can just delete that logic later if it is bad.

# Expanding Positional Embeddings

```
if max_pos > self.weight.size(0): # recompute/expand embeddings if needed
```

> The reason I haven't autoexpanded the bart positional embeddings so far is that I wanted an error for long sequences that the model would translate poorly, instead of just poor performance. But if you can like concatenate a few en-ru examples and see that performance doesn't plummet it would be good. There is also a theoretical O(seq_len^2) theoretical cost associated with passing longer documents through transformers, so we may not want to encourage longer docs/instead write tooling that, for example, uses moses SentenceSplitter to chunk documents, pass them through the model, and rejoin the results correctly. If it just works with the auto expansion hack then I'm all for it.

@stas00: I shall investigate and will follow up when I got a chance to do so
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I was hoping that someone would have a need for that and then investigate, but since nobody asked this is still a may be.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>as there has been no interest in this, closing this for now.