Using segments ids in encoder-decoder model in generate function
# 🐛 Bug

## Information

I have implemented an encoderdecoder model where both of them are bert, in encoder module i'm using segment ids (token_type_ids) and give it to model easily in this way and works correctly:
```
kwargs = {'token_type_ids':segments_tensors}
outputs = model(input_ids=encoder_input, decoder_input_ids=decoder_input, **kwargs)[0]
```
but in generate function i use below code to give segment ids:
```
kwargs = {'token_type_ids':segments_tensors}
generated = model.generate(encoder_input, decoder_start_token_id=101
num_return_sequences=16,num_beams=32, **kwargs)
```
I thought this should work but commenting or uncommenting kwargs shows me that no changes in output have happened. Is there another correct approach to give arguments of encoder module and token_type_ids through generate function or not?
`segments_tensors` are currently not implemented for `generate`. Why do you want to pass them to `generate`?<|||||>I'm working on a conditional dialogue task (knowledge grounded dialogue) in which I give the conversation history and fact sentence as inputs to the model. I want to show the model that fact sentence is different from conversation history, so I decided to use segment embedding to do this and learn different segment embeddings for history and fact sentence. in training I have no problem and pass token_type_ids as kwargs, but in inferencing phase, I have this mentioned problem. <|||||>To use `token_type_ids`, you would actually need to change the line in the generate function where the encoder is called to something like this:
`encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask, token_type_ids=model_specific_kwargs['token_type_ids'])`

or maybe better:
`encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask, token_type_ids=model_specific_kwargs.pop('token_type_ids', None))`
In case no `token_type_ids` are passed.


https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L385

<|||||>> To use `token_type_ids`, you would actually need to change the line in the generate function where the encoder is called to something like this:
> `encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask, token_type_ids=model_specific_kwargs['token_type_ids'])`
> 
> or maybe better:
> `encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask, token_type_ids=model_specific_kwargs.pop('token_type_ids', None))`
> In case no `token_type_ids` are passed.
> 
> https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L385

Thank you very much for your guidance! 🙏