| # 🧠 LoL_Build-Llama3B | |
| A fine-tuned version of the LLaMA 3.2B model using QLoRA on a custom League of Legends build suggestion dataset. This model generates champion-specific item build recommendations based on gameplay roles and current meta. | |
| --- | |
| ## 📚 Dataset | |
| - **Format**: JSONL with `prompt` and `completion` fields | |
| - **Example**: | |
| ```json | |
| { | |
| "prompt": "**Champion Patch Notes:**\n- Yone passive description changed across ranks.\n- Yone spell 2 tooltip changed across ranks.\n- Gwen hp decreased from 650 to 620.\n- Gwen armor decreased from 39 to 36.\n- Darius spell 2 updated.\n- Darius armor decreased from 39 to 37.\n- Varus spell 2 tooltip changed across ranks.\n\nYou are playing Darius (fighter/tank) on the TOP lane against Teemo (marksman/mage).\nYour team consists of Gwen, Varus, Yone, Rakan, while the enemy team includes Teemo, Nunu, Pyke, Aphelios, Poppy.\nYou have the summoner spells Flash, Ghost and the runes Conqueror, Triumph, Legend: Alacrity, Last Stand, Second Wind, Overgrowth.\nYour opponent uses Flash, Ignite and the keystone rune Press the Attack. What items should you build during the early, mid, and late game?" | |
| , | |
| "completion": "**Early Game:**\nBuy Doran's Shield\nInventory (1): Doran's Shield\n\nBuy Rejuvenation Bead\nInventory (2): Doran's Shield, Rejuvenation Bead\n\nBuy Boots\nInventory (3): Doran's Shield, Rejuvenation Bead, Boots\n\nBuy Null-Magic Mantle\nInventory (4): Doran's Shield, Rejuvenation Bead, Boots, Null-Magic Mantle\n\nBuild Mercury's Treads (fusing Boots + Null-Magic Mantle)\nInventory (3): Doran's Shield, Rejuvenation Bead, Mercury's Treads\n\nBuy Long Sword\nInventory (4): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Long Sword\n\n**Mid Game:**\nBuild Phage (fusing Ruby Crystal + Long Sword)\nInventory (4): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Phage\n\nBuy Long Sword\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Phage, Long Sword\n\nBuild Stridebreaker (fusing Tiamat + Phage + Dagger)\nInventory (4): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker\n\nBuy Ruby Crystal\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Ruby Crystal\n\nBuild Winged Moonplate (fusing Ruby Crystal)\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Winged Moonplate\n\nBuy Chain Vest\nInventory (6): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Winged Moonplate, Chain Vest\n\nBuild Dead Man's Plate (fusing Winged Moonplate + Ruby Crystal + Chain Vest)\nInventory (5): Doran's Shield, Rejuvenation Bead, Mercury's Treads, Stridebreaker, Dead Man's Plate\n\n**Late Game:**\nSell Rejuvenation Bead\nInventory (4): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate\n\nBuy Pickaxe\nInventory (5): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Pickaxe\n\nBuy Tunneler\nInventory (6): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Pickaxe, Tunneler\n\nBuild Sterak's Gage (fusing Pickaxe + Tunneler + Ruby Crystal)\nInventory (5): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage\n\nBuy Chain Vest\nInventory (6): Doran's Shield, Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest\n\nSell Doran's Shield\nInventory (5): Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest\n\nBuy Null-Magic Mantle\nInventory (6): Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest, Null-Magic Mantle\n\nBuild Negatron Cloak (fusing Null-Magic Mantle)\nInventory (6): Mercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest, Negatron Cloak\n\n**Final inventory:**\nMercury's Treads, Stridebreaker, Dead Man's Plate, Sterak's Gage, Chain Vest, Negatron Cloak" | |
| } | |
| ``` | |
| --- | |
| ## 🏋️♂️ Training Configuration | |
| | Hyperparameter | Value | | |
| |----------------------------|----------------------------------| | |
| | Base Model | unsloth/Llama-3.2-3B-bnb-4bit | | |
| | Epochs | 1 | | |
| | Max Steps | 4751 | | |
| | Batch Size | 16 | | |
| | Max Sequence Length | 1024 | | |
| | Gradient Accumulation | 1 | | |
| | Learning Rate | 2e-4 | | |
| | Weight Decay | 0.01 | | |
| | Optimizer | AdamW (8bit) | | |
| | Precision | BF16 (fallback to FP16) | | |
| | LoRA Config | r=16, alpha=32, dropout=0.05 | | |
| | Random Seed | 1515 | | |
| --- | |
| ## 📏 Evaluation Metrics | |
| Evaluated using BLEU, ROUGE-L, and Exact Match on validation prompts. | |
| | Metric | Value | | |
| |---------------|--------| | |
| | BLEU | 0.874 | | |
| | ROUGE-L | 0.930 | | |
| --- | |
| ## 📊 Final Training Metrics | |
| | Metric | Value | | |
| |---------------------------|----------------------| | |
| | Final Training Loss | 0.1162 | | |
| | Final Eval Loss | 0.1189 | | |
| | Final Grad Norm | 0.1153 | | |
| | Eval Runtime | 3341.30 sec | | |
| | Eval Samples/sec | 2.527 | | |
| | Eval Steps/sec | 0.316 | | |
| | Effective Training Steps | 4750 | | |
| --- | |
| ## ⚙️ Usage | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM | |
| tokenizer = AutoTokenizer.from_pretrained("HatimF/LoL_Build-Llama3B") | |
| model = AutoModelForCausalLM.from_pretrained("HatimF/LoL_Build-Llama3B") | |
| prompt = """**Champion Patch Notes:** | |
| Matchup: Shaco (assassin) vs Reksai (JUNGLE lane). | |
| Allies: Riven, Zoe, Cassiopeia, Nautilus. | |
| Enemies: Fiora, RekSai, Akali, Kaisa, Alistar. | |
| You are equipped with Smite, Ignite and runes Hail of Blades, Sudden Impact, Sixth Sense, Relentless Hunter, Magical Footwear, Cosmic Insight. | |
| Your opponent has Flash, Smite and keystone Conqueror. | |
| Suggest a build for early, mid, and late game.""" | |
| inputs = tokenizer(prompt, return_tensors="pt") | |
| outputs = model.generate(**inputs, max_new_tokens=100) | |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) | |
| ``` | |
| --- | |
| ## 🧠 Intended Use | |
| - **Application**: Suggests champion item builds for League of Legends based on role. | |
| - **Limitations**: | |
| - May suggest deprecated or invalid items if outdated. | |
| ## Github | |
| - https://github.com/HatimFertat/lol_builds | |