Agent trying to load remote tools when being offline
### System Info

Transformers 4.31
Python 3.11.4
Windows 10

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Code:

```
import os
os.environ['TRANSFORMERS_OFFLINE'] = '1'

from transformers import LocalAgent, AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-7b-instruct", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b-instruct")

agent = LocalAgent(model=model, tokenizer=tokenizer) # Error here

agent.run("my query");
```

Error:

```
Max retries exceeded with url: /api/spaces?author=huggingface-tools
```

### Expected behavior

To not access the remote tools.
Hi @Romainlg29 

Could you provide a complete code snippet instead of definitions like `model = ...`. Thanks in advance!<|||||>> Hi @Romainlg29
> 
> Could you provide a complete code snippet instead of definitions like `model = ...`. Thanks in advance!

Hi,

It's the following.

```
import os
os.environ['TRANSFORMERS_OFFLINE'] = '1'

from transformers import LocalAgent, AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-7b-instruct", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b-instruct")

agent = LocalAgent(model=model, tokenizer=tokenizer) # Error here

agent.run("my query");
```<|||||>cc our agent @sgugger 😆 <|||||>Agents do not work in offline mode since the prompts are fetched online and we have some tools defined on the Hub only.<|||||>If not too much work, probably not to try to connect if `os.environ['TRANSFORMERS_OFFLINE'] = '1'` and raise an error directly with a more specific message?<|||||>> Agents do not work in offline mode since the prompts are fetched online and we have some tools defined on the Hub only.

Can't we have an offline mode for the agent, where we only load our tools through additional_tools and using a custom prompt ?<|||||>@Romainlg29 You can load your tools via `additional_tools`, but the default tools are still loaded. We could add some guards around that in the future to not try to load tools from the Hub in offline mode, but it is not supported now.<|||||>Drafted a PR to add this, could you try the PR linked above? I believe it should work in offline mode as long as you have all the necessary models in the cache, and either pass custom prompts or also have the prompts in the cache. It will ignore remote tools.<|||||>> Drafted a PR to add this, could you try the PR linked above? I believe it should work in offline mode as long as you have all the necessary models in the cache, and either pass custom prompts or also have the prompts in the cache. It will ignore remote tools.

Ok, I'm going on that.