---
base_model: HuggingFaceTB/SmolLM-135M-Instruct
datasets: HumanLLMs/Human-Like-DPO-Dataset
library_name: transformers
model_name: llm-course-hw2-reward-model
tags:
- generated_from_trainer
- trl
- reward-trainer
---

# 🏆 Model Card for llm-course-hw2-reward-model  

This model is a **fine-tuned reward model** based on [HuggingFaceTB/SmolLM-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM-135M-Instruct), trained on the **[HumanLLMs/Human-Like-DPO-Dataset](https://huggingface.co/datasets/HumanLLMs/Human-Like-DPO-Dataset)** dataset.  
It has been trained using **[TRL](https://github.com/huggingface/trl)** to **evaluate and rank responses based on human preferences**, playing a crucial role in **RLHF (Reinforcement Learning from Human Feedback)** for models like **SmolLM-135M-PPO**.

---

## 📝 Overview
- **Base Model:** SmolLM-135M-Instruct
- **Fine-Tuned Dataset:** [HumanLLMs/Human-Like-DPO-Dataset](https://huggingface.co/datasets/HumanLLMs/Human-Like-DPO-Dataset)
- **Objective:** Learn to assign **higher scores** to more engaging, structured, and emotional responses.
- **Use Case:** Used in **PPO-based RLHF training** to reinforce **human-like response quality**.

### **Training Method**
- The model was fine-tuned using **Direct Preference Comparisons**:
  - Each sample contains a **chosen response** (preferred) and a **rejected response**.
  - The model **learns to assign higher rewards** to the chosen response and **lower rewards** to the rejected one.
- This reward function was used in **PPO fine-tuning** to optimize response generation.

---