File size: 2,647 Bytes
126efa4 699d7ca 126efa4 699d7ca 126efa4 699d7ca 126efa4 3664f11 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
---
datasets:
- id4thomas/emotion-prediction-comet-atomic-2020
language:
- zho
- eng
- fra
- spa
- por
- deu
- ita
- rus
- jpn
- kor
- vie
- tha
- ara
base_model:
- Qwen/Qwen2.5-3B-Instruct
---
# emotion-predictor-Qwen2.5-3B-Instruct
LLM trained to predict a character's emotional response in the given situation
* Trained to predict in a structured output format.
Prediction Performance:
| Setting | Performance by Emotion|
| --- | --- |
| Pretrained | <img src="./assets/qwen2_5-3b-baseline_perf.png" alt="baseline_perf" width="100%" /> |
| Tuned | <img src="./assets/finetuned_perf.png" alt="trained_perf" width="100%" /> |
## Quickstart
The model is trained to predict in the following schema
```
from enum import Enum
from pydantic import BaseModel
class EmotionLevel(str, Enum):
na = "na"
low = "low"
medium = "medium"
high = "high"
class EmotionLabel(BaseModel):
joy: EmotionLevel
trust: EmotionLevel
fear: EmotionLevel
surprise: EmotionLevel
sadness: EmotionLevel
disgust: EmotionLevel
anger: EmotionLevel
anticipation: EmotionLevel
class EntryResult(BaseModel):
emotion: EmotionLabel
reason: str
```
Using `outlines` package to generate structured predictions
* system prompt & user template is provided [here](./assets/inference_prompt.yaml)
```
import outlines
from outlines import models
from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("id4thomas/emotion-predictor-Qwen2.5-3B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("id4thomas/emotion-predictor-Qwen2.5-3B-Instruct")
# Initalize outlines generator
outlines_model = models.Transformers(model, tokenizer)
generator = outlines.generate.json(outlines_model, EntryResult)
# Generate
messages = [
{"role": "system", "content": system_message},
{"role": "user", "content": user_message}
]
input_text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
prediction = generator(input_text)
>>> EntryResult(emotion=EmotionLabel(joy=<EmotionLevel.na: 'na'>, ...)
```
Using endpoint loaded with vllm & OpenAI client package
* example of using vllm container is provided [here](./assets/run_vllm.sh)
```
client = OpenAI(...)
json_schema = EntryResult.model_json_schema()
completion = client.chat.completions.create(
model="id4thomas/emotion-predictor-Qwen2.5-3B-Instruct",
messages=messages,
extra_body={"guided_json": json_schema},
)
print(completion.choices[0].message.content)
``` |