README.md

# 🧠 LoL_Build-Llama3B

A fine-tuned version of the LLaMA 3.2B model using QLoRA on a custom League of Legends build suggestion dataset. This model generates champion-specific item build recommendations based on gameplay roles and current meta.

---

## 📚 Dataset

- **Format**: JSONL with `prompt` and `completion` fields
- **Example**:
  ```json
  {
    "prompt": "**Champion Patch Notes:**\n- Yone passive description changed across ranks.\n- Yone spell 2 tooltip changed across ranks.\n- Gwen hp decreased from 650 to 620.\n- Gwen armor decreased from 39 to 36.\n- Darius spell 2 updated.\n- Darius armor decreased from 39 to 37.\n- Varus spell 2 tooltip changed across ranks.\n\nYou are playing Darius (fighter/tank) on the TOP lane against Teemo (marksman/mage).\nYour team consists of Gwen, Varus, Yone, Rakan, while the enemy team includes Teemo, Nunu, Pyke, Aphelios, Poppy.\nYou have the summoner spells Flash, Ghost and the runes Conqueror, Triumph, Legend: Alacrity, Last Stand, Second Wind, Overgrowth.\nYour opponent uses Flash, Ignite and the keystone rune Press the Attack. What items should you build during the early, mid, and late game?"
    ,
    "completion": "**Early Game:**\nBuy Doran's Shield\nInventory (1): Doran's Shield\n\nBuy Rejuvenation Bead\nInventory (2): Doran's Shield, Rejuvenation Bead\n\nBuy Boots\nInventory (3): Doran's Shield, Rejuvenation Bead, Boots\n\nBuy Null-Magic Mantle\nInventory (4): Doran's Shield, Rejuvenation Bead, Boots, Null-Magic Mantle\n\nBuild Mercury's Treads (fusing Boots + Null-Magic Mantle)\nInventory (3): Doran's Shield, Rejuvenation Bead, Mercury's Treads\n\nBuy Long Sword\nInventory (4): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Long Sword\n\n**Mid Game:**\nBuild Phage (fusing Ruby Crystal + Long Sword)\nInventory (4): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Phage\n\nBuy Long Sword\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Phage, Long Sword\n\nBuild Stridebreaker (fusing Tiamat + Phage + Dagger)\nInventory (4): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker\n\nBuy Ruby Crystal\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Ruby Crystal\n\nBuild Winged Moonplate (fusing Ruby Crystal)\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Winged Moonplate\n\nBuy Chain Vest\nInventory (6): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Winged Moonplate, Chain Vest\n\nBuild Dead Man's Plate (fusing Winged Moonplate + Ruby Crystal + Chain Vest)\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Dead Man's Plate\n\n**Late Game:**\nSell Rejuvenation Bead\nInventory (4): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate\n\nBuy Pickaxe\nInventory (5): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Pickaxe\n\nBuy Tunneler\nInventory (6): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Pickaxe, Tunneler\n\nBuild Sterak's Gage (fusing Pickaxe + Tunneler + Ruby Crystal)\nInventory (5): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage\n\nBuy Chain Vest\nInventory (6): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest\n\nSell Doran's Shield\nInventory (5): Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest\n\nBuy Null-Magic Mantle\nInventory (6): Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest, Null-Magic Mantle\n\nBuild Negatron Cloak (fusing Null-Magic Mantle)\nInventory (6): Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest, Negatron Cloak\n\n**Final inventory:**\nMercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest, Negatron Cloak"
  }
  ```

---

## 🏋️‍♂️ Training Configuration

| Hyperparameter              | Value                            |
|----------------------------|----------------------------------|
| Base Model                 | unsloth/Llama-3.2-3B-bnb-4bit     |
| Epochs                     | 1                                 |
| Max Steps                  | 4751                              |
| Batch Size                 | 16                                |
| Max Sequence Length        | 1024                              |
| Gradient Accumulation      | 1                                 |
| Learning Rate              | 2e-4                              |
| Weight Decay               | 0.01                              |
| Optimizer                  | AdamW (8bit)                      |
| Precision                  | BF16 (fallback to FP16)          |
| LoRA Config                | r=16, alpha=32, dropout=0.05     |
| Random Seed                | 1515                              |

---


## 📏 Evaluation Metrics

Evaluated using BLEU, ROUGE-L, and Exact Match on validation prompts.

| Metric        | Value  |
|---------------|--------|
| BLEU          | 0.874 |
| ROUGE-L       | 0.930 |

---


## 📊 Final Training Metrics

| Metric                     | Value                |
|---------------------------|----------------------|
| Final Training Loss       | 0.1162               |
| Final Eval Loss           | 0.1189               |
| Final Grad Norm           | 0.1153               |
| Eval Runtime              | 3341.30 sec          |
| Eval Samples/sec          | 2.527                |
| Eval Steps/sec            | 0.316                |
| Effective Training Steps  | 4750                 |

---

## ⚙️ Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HatimF/LoL_Build-Llama3B")
model = AutoModelForCausalLM.from_pretrained("HatimF/LoL_Build-Llama3B")

prompt = """**Champion Patch Notes:**

Matchup: Shaco (assassin) vs Reksai (JUNGLE lane).
Allies: Riven, Zoe, Cassiopeia, Nautilus.
Enemies: Fiora, RekSai, Akali, Kaisa, Alistar.
You are equipped with Smite, Ignite and runes Hail of Blades, Sudden Impact, Sixth Sense, Relentless Hunter, Magical Footwear, Cosmic Insight.
Your opponent has Flash, Smite and keystone Conqueror.
Suggest a build for early, mid, and late game."""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## 🧠 Intended Use

- **Application**: Suggests champion item builds for League of Legends based on role.
- **Limitations**:
  - May suggest deprecated or invalid items if outdated.

## Github

- https://github.com/HatimFertat/lol_builds