AutoTokenizer behavior changing when enable_full_determinism is set to zero
### System Info

- `transformers` version: 4.20.1
- Platform: Linux-5.14.0-1045-oem-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.11.0+cu102 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@SaulLu 
However, I am quite sure this is a problem with `enable_full_determinism` rather than with tokenizers, but I don't know who to tag on that @LysandreJik 

### Reproduction

MWE:
```python
from datasets import load_dataset
from transformers import AutoTokenizer
from transformers import enable_full_determinism

# Loading tokenizer
tk = AutoTokenizer.from_pretrained("prajjwal1/bert-tiny");
tk.model_max_length = 512
def tokenize_function(examples):
    return tk(examples["text"], truncation=True, max_length=512)

# Loading dataset 
data = load_dataset('yelp_review_full')["train"].select(range(2000))
data = data.train_test_split(0.2)

data_tk = data.map(tokenize_function)  
print(max([len(b) for b in data_tk["test"]["input_ids"]])) # 512
enable_full_determinism(0)
data_tk = data.map(tokenize_function)  
print(max([len(b) for b in data_tk["test"]["input_ids"]])) # 965 !!!!!!!
enable_full_determinism(1)
data_tk = data.map(tokenize_function)  
print(max([len(b) for b in data_tk["test"]["input_ids"]])) # 512
enable_full_determinism(2)
data_tk = data.map(tokenize_function)  
print(max([len(b) for b in data_tk["test"]["input_ids"]])) # 512
```

### Expected behavior

The AutoTokenizer should systematically apply truncation, even when the seed of `enable_full_determinism` is set to 0. 
Seeds different than 0 do not seem to create problems.
 
(I suspect the seed is cast to a boolean somewhere, since the boolean of an integer is always True, except when the integer is 0).
Hi @clefourrier ,

From your description, I agree with you that the output should always be 512. However, I can't reproduce your problem (I tried with the same version of `transformers` and `torch`), I let you see my attempt to reproduce your error in this [google colab notebook](https://colab.research.google.com/drive/1R_b8u6envps2yKFrPykVunfwF8NFrzkd?usp=sharing).

I wonder if this could be caused by a cache problem for you, could you try again with:
```python
enable_full_determinism(0)
data_tk = data.map(tokenize_function, load_from_cache_file=False)  
print(max([len(b) for b in data_tk["test"]["input_ids"]]))
```

<|||||>@SaulLu 

It's not happening with `load_from_chache=False`, nor after restarting my kernel. I agree with you, must have been a cache problem (> closing). Thank you for having taken a look :hugs: