Missing adapter_config.json for PEFT Model Loading
Hello ssz1111,
I'm currently working on a project for my bachelor thesis and am very interested in using your NOVA-LLaMA-3-8B-Alpaca-5percent model, which appears to be a fantastic fine-tune of Llama 3 8B. Thank you for making it available!
I'm trying to load the model using the transformers and peft libraries in Python, as it seems to be a LoRA adapter set. My current setup looks like this (relevant part):
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig
import torch
BASE_MODEL_ID = "meta-llama/Meta-Llama-3-8B"
ADAPTER_MODEL_ID = "ssz1111/NOVA-LLaMA-3-8B-Alpaca-5percent"
# ... (Load base tokenizer and model) ...
peft_config = PeftConfig.from_pretrained(ADAPTER_MODEL_ID) # This line fails
model = PeftModel.from_pretrained(model, ADAPTER_MODEL_ID) # This line would also fail
When I attempt to load the PEFT configuration using PeftConfig.from_pretrained("ssz1111/NOVA-LLaMA-3-8B-Alpaca-5percent"), I consistently encounter the following error:
An error occurred: Can't find 'adapter_config.json' at 'ssz1111/NOVA-LLaMA-3-8B-Alpaca-5percent'
I've checked the "Files and versions" tab on the model's Hugging Face page, and it appears that the adapter_config.json file is indeed missing from the repository. The adapter_model.safetensors file (the LoRA weights) is present, but the corresponding configuration file is not.
Would it be possible for you to upload the missing adapter_config.json file to the repository or give me any tips how I may best use this model locally? This would greatly help me, and likely other users, in utilizing your excellent fine-tuned model.
Thank you for your time and assistance!
Golab