Update README.md

README.md

@@ -14,49 +14,119 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/adam-lucek/huggingface/runs/vhp5k2tx)
-# gemma-2-9b-it-lora-yt-titles
+# LoRA Adapters for Gemma-2-9B-IT on YouTube Titles
 
-This model is a fine-tuned version of [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 2.2142
+These are LoRA adapters for [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it) trained on [AdamLucek/youtube-titles](https://huggingface.co/datasets/AdamLucek/youtube-titles).
 
-## Model description
-
-More information needed
+Intended task is to tune Gemma 2 9B to generate YouTube title's more similar to popular YouTubers, data was prepped in the instruction tuned token format.
 
 ## Intended uses & limitations
 
-More information needed
-
-## Training and evaluation data
-
-More information needed
+See original model page [google/gemma-2-9b-it intented usage]https://huggingface.co/google/gemma-2-9b-it#intended-usage) for details about Gemma 2 9B usage, limitations, and ethical considerations.
+
+## Usage
+
+The below code will show you how to load and interface with the LoRA model.
+
+**Loading the Model & LoRA Adapters**
+```python
+from peft import PeftConfig, PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+import torch
+
+# Load the Pre Trained Model
+model = AutoModelForCausalLM.from_pretrained("google/gemma-2-9b-it", 
+                                             quantization_config=BitsAndBytesConfig(load_in_8bit=True), 
+                                             device_map="auto"
+                                             ).eval()
+# Load the Tokenizer
+tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b-it")
+
+# Attach LoRA Adapters to Pre Trained Model
+model = PeftModel.from_pretrained(model, "AdamLucek/gemma-2-9b-it-lora-yt-titles", adapter_name="youtube_titles")
+```
+**Inference**
+
+```python
+topic = "huggingface AI models"
+messages = [
+    {"role": "user", "content": f"Create a YouTube title about {topic}"}
+]
+
+# Apply chat template and prepare inputs
+text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
+inputs = tokenizer(text, return_tensors="pt")
+inputs = {k: v.to("cuda") for k, v in inputs.items()}
+
+# Generate outputs
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256,
+    do_sample=True,
+    top_p=0.95,
+    temperature=0.1,
+    repetition_penalty=1.2,
+    eos_token_id=tokenizer.eos_token_id
+)
+  
+# Decode outputs
+decoded = tokenizer.decode(outputs[0])
+
+print(decoded)
+```
 
 ## Training procedure
 
 ### Training hyperparameters
 
-The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 4
-- eval_batch_size: 4
-- seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 16
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: cosine
-- lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 3.0
+Trained on a single a6000 using the following script
+
+```
+python \
+    examples/scripts/sft.py \
+    --model_name_or_path="google/gemma-2-9b-it" \
+    --dataset_name="AdamLucek/youtube-titles" \
+    --dataset_text_field="gemma2_9b_it_format" \
+    --per_device_train_batch_size=4 \
+    --per_device_eval_batch_size=4 \
+    --gradient_accumulation_steps=4 \
+    --max_grad_norm=1.0 \
+    --learning_rate=5e-5 \
+    --weight_decay=0.01 \
+    --lr_scheduler_type="cosine" \
+    --warmup_ratio=0.1 \
+    --report_to="wandb" \
+    --bf16 \
+    --max_seq_length=2048 \
+    --lora_r=16 \
+    --lora_alpha=32 \
+    --lora_target_modules q_proj k_proj v_proj o_proj \
+    --load_in_8bit \
+    --use_peft \
+    --attn_implementation="eager" \
+    --logging_steps=1 \
+    --eval_strategy="steps" \
+    --eval_steps=200 \
+    --save_strategy="steps" \
+    --save_steps=250 \
+    --output_dir="models/gemma2" \
+    --hub_model_id="gemma-2-9b-it-lora-yt-titles" \
+    --push_to_hub \
+    --num_train_epochs=3
+```
 
 ### Training results
 
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/adam-lucek/huggingface/runs/vhp5k2tx)
+
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
 | 2.2556        | 0.7619 | 200  | 2.0945          |
 | 2.1866        | 1.5238 | 400  | 2.0988          |
 | 2.3421        | 2.2857 | 600  | 2.2142          |
 
+It achieves the following results on the evaluation set:
+- Loss: 2.2142
 
 ### Framework versions