2 RAG test failures
The two failures here look related to recent tokenizer changes:

https://github.com/huggingface/transformers/runs/1236753957?check_suite_focus=true

```
=========================== short test summary info ============================

FAILED tests/test_modeling_rag.py::RagModelIntegrationTests::test_rag_sequence_generate_batch
FAILED tests/test_modeling_rag.py::RagModelIntegrationTests::test_rag_token_generate_batch
```
@sshleifer - thanks a lot for the issue! 

This error seems related to https://github.com/huggingface/transformers/issues/7690#issuecomment-707382445. 