unable to load cache when network is unavailable
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.8.2
- Platform: Linux-4.4.0-101-generic-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyTorch version (GPU?): 1.8.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

## Information

Cannot load cache when ETag is unavailable (because of network connection issue etc.), because the filename of cache is something like `xxxxxxx(.<ETag>)`, when ETag is unavailable, the program tries to find something like `xxxxxxx` (without ETag as suffix), which won't succeed.

## To reproduce

Write some codes simply load a pre-trained tokenizer:

```python
from transformers import BertTokenizer

BertTokenizer.from_pretrained('bert-base-uncased')
```

Run the code above when network is available, wait for the program complete downloading the model and saving the cache. Then cutoff network connection and run the code above again, it will raise following exception:

```
ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.
```


## Expected behavior

The program loads the cache downloaded previously when network is unavailable.


I might found the reason: when network is unavailable, the program tries to find `added_tokens.json`, however this was not downloaded when network is available.<|||||>> 
> 
> I might found the reason: when network is unavailable, the program tries to find `added_tokens.json`, however this was not downloaded when network is available.

Because the file `added_tokens.json` does not actually exists for `bert-base-uncased` model, so the program should not try to find this file when load from cache. I think maybe the cache should also maintain a list of all files.<|||||>This is not a issue related to the ETag, sorry for the misleading description at the beginning.<|||||>Have you find any solutions for this?

<|||||>@zhouyanxin282446 No, but maybe it's possible to create some empty dummy files, I haven't checked yet.<|||||>When you don't have internet access, you should specify `local_files_only` to `from_pretrained` so that the package only tries to fetch local files:

```py
BertTokenizer.from_pretrained('bert-base-uncased', local_files_only=True)
```<|||||>> When you don't have internet access, you should specify `local_files_only` to `from_pretrained` so that the package only tries to fetch local files:
> 
> ```python
> BertTokenizer.from_pretrained('bert-base-uncased', local_files_only=True)
> ```

This works, thanks.

Actually, in my case, I'm only able to access hugging face stably when I'm using a proxy, but I don't want to enable the proxy all the time, so I'm used to firstly downloading the model with proxy, and running my code without proxy afterwards, without modifying the code (by setting `local_files_only`). How about trying local files after encountering a network error when `local_files_only` is `False`? Sounds reasonable doesn't it.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.