File size: 3,429 Bytes
5d2c9e3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
---
license: mit
base_model: Qwen/Qwen2.5-3B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
- lora
- transformers
- korean
- npc
- game-ai
---
# npc_LoRA
**npc_LoRA** is a LoRA adapter built on top of [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct), designed to generate emotionally rich, context-aware dialogue for non-player characters (NPCs) in Korean-language game environments.
This project is part of a portfolio for industrial service roles in AI and game development, showcasing practical model design, multi-head training, and real-world integration strategies.
## 🧠 Model Architecture
- **Base model**: Qwen2.5-3B-Instruct
- **Adapter type**: LoRA (via PEFT)
- **Language**: Korean
- **Task**: Text generation with auxiliary heads
- **Heads added**:
- `delta_head`: Predicts 2D continuous values for narrative state change
- `flag_head`: Predicts 3 or more binary flags for game logic triggers
## 🏗️ Training Setup
- **Environment**: Google Colab with A100 GPU
- **Quantization**: 4-bit (nf4) via BitsAndBytes
- **Batch size**: 2 (gradient accumulation: 8)
- **Epochs**: 6
- **Losses**:
- Language modeling (CrossEntropy)
- Delta prediction (MSE)
- Flag prediction (BCE)
## 📜 Prompt Format
```text
<SYS>
NPC_ID=...
TAGS:
location=...
quest_stage=...
relationship=...
trust=...
npc_mood=...
player_reputation=...
style=...
REQUIRE:
...
FORMAT:
<RESPONSE>...</RESPONSE>
<DELTA ...>
<FLAG ...>
</SYS>
<CTX>
player: ...
npc: ...
</CTX>
<PLAYER>...
<NPC>
```
## 🔍 Inference Example
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch.nn as nn
BASE_MODEL = "Qwen/Qwen2.5-3B-Instruct"
ADAPTER_PATH = "minjae/npc_LoRA"
tokenizer = AutoTokenizer.from_pretrained(ADAPTER_PATH, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER_PATH)
# Add heads
hidden_size = model.config.hidden_size
model.delta_head = nn.Linear(hidden_size, 2).to(model.device)
model.flag_head = nn.Linear(hidden_size, 3).to(model.device)
prompt = "<SYS>...<CTX>...<PLAYER>...<NPC>"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model(**inputs, output_hidden_states=True)
gen_ids = model.generate(**inputs, max_new_tokens=100)
generated_text = tokenizer.decode(gen_ids[0], skip_special_tokens=True)
last_hidden = outputs.hidden_states[-1][:, -1, :]
delta = model.delta_head(last_hidden)
flag = model.flag_head(last_hidden)
print("Response:", generated_text)
print("Delta:", delta)
print("Flags:", torch.sigmoid(flag))
```
## 🧩 Use Cases
- NPC dialogue generation in Korean RPGs
- Emotionally adaptive storytelling
- Game logic trigger prediction (e.g., quest progression, item handoff)
## 📁 Repository Structure
```
npc_LoRA/
├── lora-output-jason-mom-head/ # LoRA adapter files
├── README.md
```
## 📌 Notes
- Adapter is optimized for Korean-language prompts and multi-turn dialogue.
- Designed to integrate with game engines or AI-driven simulation platforms.
- Compatible with Hugging Face Spaces (CPU/GPU) and local inference.
## 📜 License
MIT
## 👤 Author
Created by **Minjae**
Portfolio: [GitHub Profile](https://github.com/m97j)
Contact: [[email protected]]
|