`local_files_only=True` not work
### System Info

torch                   1.12.1
transformers            4.21.1

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I have downloaded the pretrained weights and the model worked well.
I am confused by the long loading time (~25s on a SSD) when using the `from_pretrained` API, and I set `local_files_only=True` to disable connections.
But I still found the function called a http request and it spent many seconds on my desktop without Internet.
Here is my log file:

> File "/disk1/fewshot/anaconda3/envs/pet/lib/python3.9/site-packages/transformers/utils/hub.py", line 284, in cached_path
>     output_path = get_from_cache(
>   File "/disk1/fewshot/anaconda3/envs/pet/lib/python3.9/site-packages/transformers/utils/hub.py", line 501, in get_from_cache
>     r = requests.head(url, headers=headers, allow_redirects=False, proxies=proxies, timeout=etag_timeout)


### Expected behavior

When set `local_files_only=True`, it should disable connections to the website.
how did you solve it?