Environment info


transformers version: 4.9.0
Platform: Linux-5.4.104±x86_64-with-Ubuntu-18.04-bionic
Python version: 3.7.11
PyTorch version (GPU?): 1.9.0+cu102 (True)
Tensorflow version (GPU?): 2.5.0 (True)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: No
Using distributed or parallel set-up in script?: No


Who can help
@vasudevgupta

Information
Model I am using BigBirdTokenizerFast.from_pretrained(‘google/bigbird-roberta-base’)
The likely problem is when you load the pre-trained tokenizer and check its eos and bos token mapping, it turns out to be opposite of what is expected, namely:
bos_token = 
eos_token = 

To reproduce

  

      gist.github.com
  

  
    https://gist.github.com/nabito/64155ccff96f2fd08311374d5bf2bb6f 1

  bigbird-playground.ipynb
  {
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "BigBird Playground.ipynb",
      "provenance": [],
      "machine_shape": "hm",
      "authorship_tag": "ABX9TyO9QaJHqhGtlYbgYV84ucT8",
      "include_colab_link": true
    This file has been truncated. show original




  

  
    
    
  

  



Expected behavior
According to the implementation of BigBirdTokenizerFast the default special token mapping should be:
bos_token = 
eos_token =