HTTPS request to model repo despite local_files_only=T
### System Info

- `transformers` version: 4.9.2
- Platform: Linux-5.4.0-109-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.10.0a0+git36449ea (True)
- Tensorflow version (GPU?): 2.4.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: none


### Who can help?

@mrm8488 
I am trying to to Q&A in a very large text so I chunk the text and do Q&A in smaller chunks. 
I have a pipeline using this model embedded in a function that I call in a loop.
```
  my_guru <- function(my_sents, my_topk, my_question) {
    sequence_length <- lengths(gregexpr("\\W+", my_sents)) + 1
    print(sequence_length)
    assert("Sequence length < 4096", sequence_length < 4096)
    text <- reticulate::import("tensorflow_text")
    transformers <- reticulate::import("transformers")
    torch <- reticulate::import("torch")
    
    model <- transformers$AutoModelForQuestionAnswering$from_pretrained("mrm8488/longformer-base-4096-finetuned-squadv2",
                                                                        #low_cpu_mem_usage=FALSE,
                                                                        local_files_only=T)
    tokenizer <- transformers$AutoTokenizer$from_pretrained("mrm8488/longformer-base-4096-finetuned-squadv2",
                                                            truncation = FALSE,
                                                            padding='max_length',
                                                            local_files_only=T)
    guru <- transformers$pipeline("question-answering", 
                                  model=model, 
                                  tokenizer=tokenizer,
                                  device=0L)
    
    answers <- guru(context = my_sents, question = my_question, top_k = my_topk)
    rm(tokenizer)
    gc()
    torch$cuda$empty_cache()
    return(answers)
  }
````

Since I am using local_files_only=T. I should expect it to run over the days and complete without going to internet. However after looping a few thousand times, It generates an error and crashes out

```
/usr/local/lib/python3.8/dist-packages/transformers/pipelines/question_answering.py:316: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:198.)
  fw_args = {k: torch.tensor(v, device=self.device) for (k, v) in fw_args.items()}
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/mrm8488/longformer-base-4096-finetuned-squadv2
```



### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

set local files to TRUE
repeat the pipeline over thousands of times

### Expected behavior

The script should not send a request to huggingface when local files is set to TRUE
You need to upgrade your version of Transformers as it's pretty old and many bugs with the cache system have been fixed since then.<|||||>Thank you. Upgrade fixes the problem