README.md

---
license: mit
datasets:
- eagle0504/openai-gsm8k-enhanced-using-together-ai-deepseek-train8k-test1k-v1
language:
- en
base_model:
- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
library_name: transformers
tags:
- fine-tuned
- unsloth
- trl
- grpo
- deepseek
- gsm8k
- reasoning
---

# **DeepSeek-R1-Distill-Qwen-1.5B Fine-Tuned on GSM8K with Chain-of-Thought Augmentation**

## **Model Overview**
This model is a fine-tuned version of **DeepSeek-R1-Distill-Qwen-1.5B**, trained on the **OpenAI GSM8K dataset**, augmented with **Chain-of-Thought (CoT) reasoning** using **DeepSeek-V3**. The fine-tuning process enhances the model’s **mathematical problem-solving abilities**, allowing it to provide **step-by-step solutions** with deeper reasoning.

### **🔹 Key Features**
- **Base Model**: DeepSeek-R1-Distill-Qwen-1.5B  
- **Fine-Tuned On**: GSM8K dataset with DeepSeek-V3-enhanced reasoning  
- **Improved Mathematical Reasoning**: Generates detailed step-by-step CoT explanations  
- **Optimized for GRPO Training**: Trained using `trl` and `unsloth` for efficient fine-tuning  

---

## **📊 Dataset & Training Details**
- **Dataset**: `eagle0504/openai-gsm8k-enhanced-using-together-ai-deepseek-train8k-test1k-v1`
  - **8K train samples**, **1K test samples**
  - Contains **question**, **answer**, and **CoT reasoning**
- **Training Methodology**:
  - Used **Guided Reinforcement Policy Optimization (GRPO)** via `trl`
  - Applied **gradient accumulation** to manage larger batch sizes
  - Integrated **DeepSeek-V3 augmentation** for enhanced logical reasoning
- **Fine-tuning Tools**:
  - **Unsloth** for memory-efficient Llama-based tuning
  - **Hugging Face Transformers** for model training

For those interested in replicating the fine-tuning process, I have shared an **updated Colab notebook** 📓:  
🔗 [Colab Notebook](https://colab.research.google.com/drive/1HV0YkyiTD55j1xLRBHwJ_q3ex82W5EXr?usp=sharing)

You will need:  
✅ Hugging Face Token  
✅ Together.AI API Key  
✅ Unsloth Package  

---

## **🚀 How to Run the Model (Mac via `llama.cpp`)**
Yes! You can run this model **locally on macOS** using `llama.cpp`.  

### **1️⃣ Install Homebrew (If Not Installed)**
```sh
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```
Then add Homebrew to your PATH:
```sh
echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile
eval "$(/opt/homebrew/bin/brew shellenv)"
```

---

### **2️⃣ Install `llama.cpp`**
```sh
brew install llama.cpp
```

---

### **3️⃣ Run the Model with `llama-cli`**
```sh
llama-cli -hf eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced-gguf:Q8_0
```

---

### **4️⃣ Alternative: Run Locally via GGUF**
```sh
mkdir -p ~/llama_models && cd ~/llama_models
wget https://huggingface.co/eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced-gguf/resolve/main/Q8_0.gguf
llama-cli -m ~/llama_models/Q8_0.gguf --interactive
```

---

## **📌 How to Use Model via Python (`transformers`)**
You can load the model with **Hugging Face Transformers**:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "A farmer has 24 apples. He gives 6 to each of his 3 children. How many does he have left?"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_length=200)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

---

## **🔬 Expected Performance**
Compared to the base **DeepSeek-R1-Distill-Qwen-1.5B**, this fine-tuned model:
- Provides **more detailed Chain-of-Thought (CoT) explanations** for GSM8K problems.
- Improves **logical reasoning and step-by-step answer formulation**.
- Generates **clearer, more structured solutions**, making it **ideal for educational use**.

---

## **🗂 Model Hosting & License**
📌 **Model on Hugging Face Hub**:  
👉 **[eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced](https://huggingface.co/eagle0504/deepseek-r1-qwen-1.5b-gsm8k-enhanced)**  

📜 **License**: MIT License – Open for modification and distribution.

---

If you have **feedback or ideas for improvement**, feel free to reach out! 🚀🔥  

#AI #MachineLearning #DeepSeek #GSM8K #LLM #ChainOfThought #HuggingFace #GRPO #Reasoning
```