General Update Question for LLMs
I see this model was updated, do I have to delete the previous version when using oobabooga to download the model or will it update the required files automatically? Oobabooga appears to have downloaded two small files, but no big model, Im guessing that the model file was unchanged and the config files were updated?
That's correct. I just updated the config.json to match changes in the upstream repo. The changes may not even make a difference in text-generation-webui as it has independent controls for those parameters. But yes you've done all you need to do and now have the latest files.
text-generation-webui's downloader only downloads changes, so it's always safe to trigger a download of an existing repo - you will get anything new, and won't re-download anything you already have.
@TheBloke Hiya
I just tested the updated model and was getting an error related to the config.json file. I fixed it using this modification:
{
"architectures": [
"LlamaForCausalLM"
],
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 5120,
"initializer_range": 0.02,
"intermediate_size": 13824,
"max_position_embeddings": 2048,
"model_type": "llama",
"num_attention_heads": 40,
"num_hidden_layers": 40,
"pad_token_id": 0,
"rms_norm_eps": 1e-05,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.30.2",
"use_cache": true,
"vocab_size": 32000
}