Model Not Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "openchat/openchat_3.5"
model = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(repo)
Error:
```
KeyError Traceback (most recent call last)
Cell In[21], line 5
2 import torch
4 repo = "openchat/openchat_3.5"
----> 5 model = AutoModelForCausalLM.from_pretrained(
6 repo, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
7 tokenizer = AutoTokenizer.from_pretrained(repo)
File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:527, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
524 if kwargs.get("quantization_config", None) is not None:
525 _ = kwargs.pop("quantization_config")
--> 527 config, kwargs = AutoConfig.from_pretrained(
528 pretrained_model_name_or_path,
529 return_unused_kwargs=True,
530 trust_remote_code=trust_remote_code,
531 code_revision=code_revision,
532 _commit_hash=commit_hash,
533 **hub_kwargs,
534 **kwargs,
535 )
537 # if torch_dtype=auto was passed here, ensure to pass it on
538 if kwargs_orig.get("torch_dtype", None) == "auto":
File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1041, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
1039 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
1040 elif "model_type" in config_dict:
-> 1041 config_class = CONFIG_MAPPING[config_dict["model_type"]]
1042 return config_class.from_dict(config_dict, **unused_kwargs)
1043 else:
1044 # Fallback: use pattern matching on the string.
1045 # We go from longer names to shorter names to catch roberta before bert (for instance)
File /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:734, in _LazyConfigMapping.getitem(self, key)
732 return self._extra_content[key]
733 if key not in self._mapping:
--> 734 raise KeyError(key)
735 value = self._mapping[key]
736 module_name = model_type_to_module_name(key)
KeyError: 'mistral'
Hello, this error strictly indicate that you have to upgrade your version of transformers library. simple pip install --upgrade transformers should help,
Hello, this error strictly indicate that you have to upgrade your version of transformers library. simple pip install --upgrade transformers should help,
Tried many a times, its an issue which is occurring while using notebook like kaggle, paperspace
(…)esolve/main/pytorch_model.bin.index.json: 0%| | 0.00/23.9k [00:00<?, ?B/s]
Downloading shards: 0%| | 0/2 [00:00<?, ?it/s]
pytorch_model-00001-of-00002.bin: 0%| | 0.00/9.94G [00:00<?, ?B/s]
pytorch_model-00002-of-00002.bin: 0%| | 0.00/4.54G [00:00<?, ?B/s]
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
(…)t_3.5/resolve/main/tokenizer_config.json: 0%| | 0.00/1.38k [00:00<?, ?B/s]
tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s]
(…)openchat_3.5/resolve/main/tokenizer.json: 0%| | 0.00/1.80M [00:00<?, ?B/s]
(…)nchat_3.5/resolve/main/added_tokens.json: 0%| | 0.00/53.0 [00:00<?, ?B/s]
(…)3.5/resolve/main/special_tokens_map.json: 0%| | 0.00/491 [00:00<?, ?B/s]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Works fine for me with transformers version 4.35.2
(…)esolve/main/pytorch_model.bin.index.json: 0%| | 0.00/23.9k [00:00<?, ?B/s] Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] pytorch_model-00001-of-00002.bin: 0%| | 0.00/9.94G [00:00<?, ?B/s] pytorch_model-00002-of-00002.bin: 0%| | 0.00/4.54G [00:00<?, ?B/s] Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] (…)t_3.5/resolve/main/tokenizer_config.json: 0%| | 0.00/1.38k [00:00<?, ?B/s] tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s] (…)openchat_3.5/resolve/main/tokenizer.json: 0%| | 0.00/1.80M [00:00<?, ?B/s] (…)nchat_3.5/resolve/main/added_tokens.json: 0%| | 0.00/53.0 [00:00<?, ?B/s] (…)3.5/resolve/main/special_tokens_map.json: 0%| | 0.00/491 [00:00<?, ?B/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.Works fine for me with transformers version 4.35.2
I was using 4.34.1