|
|
--- |
|
|
license: mit |
|
|
tags: |
|
|
- svector |
|
|
- theta-35-mini |
|
|
- theta |
|
|
--- |
|
|
|
|
|
# Theta-35-mini |
|
|
|
|
|
**A lightweight, high-efficiency reasoning model distilled from Theta-35.** |
|
|
**Theta-35-Mini** is a compact 3B parameter language model developed by **SVECTOR**, built on the Qwen architecture and trained using **Group Relative Policy Optimization (GRPO)**. It is the smaller sibling of our flagship **Theta-35** model (33B parameters), offering efficient performance for resource-constrained environments. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔍 Overview |
|
|
|
|
|
- **Architecture**: Based on Qwen2-style transformer blocks |
|
|
- **Training Objective**: Autoregressive next-token prediction |
|
|
- **Technique**: Trained using **Group Relative Policy Optimization (GRPO)** – a reinforcement learning optimization strategy enabling fine-grained control and alignment |
|
|
- **Size**: 3 billion parameters |
|
|
- **Parent Model**: [Theta-35 (33B)](https://huggingface.co/SVECTOR-CORPORATION/Theta-35) |
|
|
|
|
|
|
|
|
## 🚀 Model Highlights |
|
|
|
|
|
- ✅ **Compact and Capable**: Achieves strong performance despite its small size |
|
|
- ⚙️ **GRPO-trained**: Trained with Group Relative Policy Optimization for better alignment, coherence, and efficiency |
|
|
- 💡 **Low-latency Inference**: Ideal for edge and on-device applications |
|
|
## 📦 How to Use |
|
|
|
|
|
Install dependencies: |
|
|
|
|
|
```bash |
|
|
pip install transformers |
|
|
``` |
|
|
|
|
|
Run model in Python: |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini") |
|
|
model = AutoModelForCausalLM.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini") |
|
|
|
|
|
# Prompt input |
|
|
inputs = tokenizer("Once upon a time", return_tensors="pt") |
|
|
|
|
|
# Generate output |
|
|
outputs = model.generate(**inputs, max_length=100, temperature=0.7) |
|
|
|
|
|
# Decode and print |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📄 License |
|
|
|
|
|
This model is released under the **MIT License**. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🏢 About SVECTOR |
|
|
|
|
|
🔗 Visit us at [svector.co.in](https://www.svector.co.in) |
|
|
|
|
|
--- |
|
|
|
|
|
## 🙌 Acknowledgements |
|
|
|
|
|
- DeepSeek GRPO Paper |
|
|
- Qwen2 Architecture |
|
|
--- |
