metadata

license: mit
tags:
  - svector
  - theta-35-mini
  - theta

Theta-35-mini

A lightweight, high-efficiency reasoning model distilled from Theta-35. Theta-35-Mini is a compact 3B parameter language model developed by SVECTOR, built on the Qwen architecture and trained using Group Relative Policy Optimization (GRPO). It is the smaller sibling of our flagship Theta-35 model (33B parameters), offering efficient performance for resource-constrained environments.

🔍 Overview

Architecture: Based on Qwen2-style transformer blocks
Training Objective: Autoregressive next-token prediction
Technique: Trained using Group Relative Policy Optimization (GRPO) – a reinforcement learning optimization strategy enabling fine-grained control and alignment
Size: 3 billion parameters
Parent Model: Theta-35 (33B)

🚀 Model Highlights

✅ Compact and Capable: Achieves strong performance despite its small size
⚙️ GRPO-trained: Trained with Group Relative Policy Optimization for better alignment, coherence, and efficiency
💡 Low-latency Inference: Ideal for edge and on-device applications

📦 How to Use

Install dependencies:

pip install transformers

Run model in Python:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Force use of the slow tokenizer to avoid tokenizer.json issues
tokenizer = AutoTokenizer.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini", use_fast=False)
model = AutoModelForCausalLM.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini")

inputs = tokenizer("Once upon a time", return_tensors="pt")

outputs = model.generate(**inputs, max_length=100, temperature=0.7)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📄 License

This model is released under the MIT License.

🏢 About SVECTOR

🔗 Visit us at svector.co.in

🙌 Acknowledgements

DeepSeek GRPO Paper
Qwen2 Architecture