Load BERT+GPT2 in EncoderDecoder
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
I am working on modelling an EncoderDecoderModel using weights of BERT and GPT2, after going through lots of repo and issues I found that currently its not possible, but I found a model card that has used this model of BERT+GPT2 on dataset cnn-dailymail [here](https://huggingface.co/patrickvonplaten/bert2gpt2-cnn_dailymail-fp16). I would like to know in which version of transformers is that possible and one thing more that there was an attribute passed in `TrainingArguments` module, that was `predict_from_generate`, I can't find that in `transformers`: 3.1.0, 3.0.2, 2.11.0, please clear me in which version does these parameters constitute.

@patrickvonplaten Please answer my query
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
Hi @AmbiTyga ,
Bert2GPT2 is available in the latest release, however `predict_from_generate` from generate is not yet added in `Trainer`. 
You can set `predict_from_generate` to `False`  and `comput_metrics` to `None` if you done't need generative metrics (ROUGE etc) at training time. 

If you want to use `predict_from_generate` from generate then you'll need to install transformers from this fork.
https://github.com/huggingface/transformers/tree/more_general_trainer_metric<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
