# GPT-2 DPO Fine-Tuned Model

This repository contains a fine-tuned **GPT-2** model trained using **Direct Preference Optimization (DPO)** on preference-based data.

## Model Details
- **Base Model:** GPT-2
- **Fine-tuned on:** Preference optimization dataset
- **Training Method:** Direct Preference Optimization (DPO)
- **Hyperparameters:**
  - Learning Rate: `1e-3`
  - Batch Size: `8`
  - Epochs: `5`
  - Beta: `0.1`

## Dataset
The dataset used for training is **`Dahoas/static-hh`**, a publicly available dataset on Hugging Face, designed for **human preference optimization**. It consists of multiple prompts along with corresponding **chosen** and **rejected** responses.

## Usage
Load the model and tokenizer from Hugging Face:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "PhuePwint/dpo_gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Generate response
prompt = "What is the purpose of life?"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
output = model.generate(input_ids, max_length=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))