Get following error with EncoderDecoder model: TypeError: forward() got an unexpected keyword argument 'use_cache'
Hi

I am trying to create an EncoderDecoder model where i want to use a pre-trained encoder model and initialise decoder from scratch.  Following the code snippet.

----------------------------------------------------------
encoder = AutoModel.from_pretrained('bert-base-uncased')
decoder_config = BertConfig(vocab_size = vocabsize,
                    max_position_embeddings = max_length, 
                    num_attention_heads = num_attention_heads,
                    num_hidden_layers = num_hidden_layers,
                    hidden_size = hidden_size,
                    type_vocab_size = 1,
                    is_decoder=True)    

decoder = BertForMaskedLM(config=decoder_config)
model = EncoderDecoderModel(encoder=encoder, decoder=decoder)
----------------------------------------------------------
Model gets built without any errors but when i try to make a forward pass, i get the error:
TypeError: forward() got an unexpected keyword argument 'use_cache'.

Following is the dummy forward pass function

outputs = model(input_ids=input_ids, decoder_input_ids=input_ids, labels=input_ids)


Hi @mandareln 

You should use `BertLMHeadModel` class if you want to use bert as decoder, here you are using `BertForMaskedLM` which is the reason for this error as it does not have the `use_cache` argument.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.