TFEncoderDecoder
# 🚀 Feature request

Tensorflow version for the `EncoderDecoder` class for sequence to sequence text generation.

## Motivation
I am replicating [this](https://arxiv.org/pdf/1907.12461.pdf) paper which studies several combinations of encoder and decoder with pretrained checkpoints. Most of it is implemented very nicely in the current API, but is not available for tensorflow. The closest thing I found was `TFT5ForConditionalGeneration` which works nicely. Are there any plans to extend this for other pretrained models such as Bert, Roberta, GPT(2)?

## Your contribution
Happy to work on a PR if someone can provide some pointers on where to start and best practices to include new models in the existing API.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
