---
base_model:
- meta-llama/Llama-3.2-1B
---
# 🦙 LoRA Implementation of LLaMA

This model is a **Low-Rank Adaptation (LoRA)** of the **LLaMA 3.2-1B** model.

 **Original size**: ~1 Billion parameters  
 **LoRA fine-tuned parameters**: ~86,000  
 **Result**: Achieves similar performance on many prompts with dramatically reduced trainable weights.

---

### About

This model was fine-tuned using the [PEFT (Parameter-Efficient Fine-Tuning)](https://github.com/huggingface/peft) library and Hugging Face's `transformers`.

- **Base Model**: `meta-llama/Llama-3.2-1B`

### Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("your-username/Llama-3.2-1B_LoRA")
tokenizer = AutoTokenizer.from_pretrained("your-username/Llama-3.2-1B_LoRA")

inputs = tokenizer("Tell me something interesting about space.", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))