---
license: mit
base_model: Qwen/Qwen2.5-3B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
- lora
- transformers
- korean
- npc
- game-ai
---

# npc_LoRA

**npc_LoRA** is a LoRA adapter built on top of [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct), designed to generate emotionally rich, context-aware dialogue for non-player characters (NPCs) in Korean-language game environments.

This project is part of a portfolio for industrial service roles in AI and game development, showcasing practical model design, multi-head training, and real-world integration strategies.

## 🧠 Model Architecture

- **Base model**: Qwen2.5-3B-Instruct
- **Adapter type**: LoRA (via PEFT)
- **Language**: Korean
- **Task**: Text generation with auxiliary heads
- **Heads added**:
  - `delta_head`: Predicts 2D continuous values for narrative state change
  - `flag_head`: Predicts 3 or more binary flags for game logic triggers

## 🏗️ Training Setup

- **Environment**: Google Colab with A100 GPU
- **Quantization**: 4-bit (nf4) via BitsAndBytes
- **Batch size**: 2 (gradient accumulation: 8)
- **Epochs**: 6
- **Losses**:
  - Language modeling (CrossEntropy)
  - Delta prediction (MSE)
  - Flag prediction (BCE)

## 📜 Prompt Format

```text
<SYS>
NPC_ID=...
TAGS:
 location=...
 quest_stage=...
 relationship=...
 trust=...
 npc_mood=...
 player_reputation=...
 style=...
REQUIRE:
 ...
FORMAT:
 <RESPONSE>...</RESPONSE>
 <DELTA ...>
 <FLAG ...>
</SYS>
<CTX>
player: ...
npc: ...
</CTX>
<PLAYER>...
<NPC>
```

## 🔍 Inference Example

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch.nn as nn

BASE_MODEL = "Qwen/Qwen2.5-3B-Instruct"
ADAPTER_PATH = "minjae/npc_LoRA"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER_PATH, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER_PATH)

# Add heads
hidden_size = model.config.hidden_size
model.delta_head = nn.Linear(hidden_size, 2).to(model.device)
model.flag_head = nn.Linear(hidden_size, 3).to(model.device)

prompt = "<SYS>...<CTX>...<PLAYER>...<NPC>"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model(**inputs, output_hidden_states=True)
    gen_ids = model.generate(**inputs, max_new_tokens=100)
    generated_text = tokenizer.decode(gen_ids[0], skip_special_tokens=True)

    last_hidden = outputs.hidden_states[-1][:, -1, :]
    delta = model.delta_head(last_hidden)
    flag = model.flag_head(last_hidden)

print("Response:", generated_text)
print("Delta:", delta)
print("Flags:", torch.sigmoid(flag))
```

## 🧩 Use Cases

- NPC dialogue generation in Korean RPGs
- Emotionally adaptive storytelling
- Game logic trigger prediction (e.g., quest progression, item handoff)

## 📁 Repository Structure

```
npc_LoRA/
├── lora-output-jason-mom-head/         # LoRA adapter files
├── README.md
```

## 📌 Notes

- Adapter is optimized for Korean-language prompts and multi-turn dialogue.
- Designed to integrate with game engines or AI-driven simulation platforms.
- Compatible with Hugging Face Spaces (CPU/GPU) and local inference.

## 📜 License

MIT

## 👤 Author

Created by **Minjae**  
Portfolio: [GitHub Profile](https://github.com/m97j)  
Contact: [mmnkjiae@gmail.com]