---
license: apache-2.0
datasets:
- open-r1/Mixture-of-Thoughts
language:
- en
base_model:
- Qwen/Qwen3-0.6B
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation-inference
- code
- moe
- math
---

![1.png](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/C2G_EMWOcgvjmKz1Ad5PU.png)

# **Nenque-MoT-0.6B-Elite14**

> **Nenque-MoT-0.6B-Elite14** is a compact, high-efficiency model tailored for **mathematical reasoning**, **code generation**, and **structured technical inference**. Fine-tuned from **Qwen3-0.6B** using the **MoT (Mixture of Thoughts) dataset**—with a focus on **math expert clusters**—this model delivers strong symbolic performance in low-resource environments. Despite its **0.6B parameter** size, it offers elite-level precision across STEM and multilingual technical domains.

> [!note]
GGUF: [https://huggingface.co/prithivMLmods/Nenque-MoT-0.6B-Elite14-GGUF](https://huggingface.co/prithivMLmods/Nenque-MoT-0.6B-Elite14-GGUF)

---

## **Key Features**

1. **MoT Fine-Tuning on Math Expert Clusters**
   Trained on a curated **Mixture of Thoughts (MoT)** dataset emphasizing symbolic mathematics, code reasoning, and problem-solving, enhancing precision in structured tasks.

2. **Elite Mathematical Reasoning**
   Excels in solving algebraic equations, calculus, and symbolic logic step-by-step—suitable for education, competitions, and STEM support tools.

3. **Compact Code Assistant**
   Generates concise, explainable code in Python, JavaScript, and others—ideal for code tutoring, bug diagnosis, and fast prototyping.

4. **Structured Output Generation**
   Supports generation in **Markdown**, **JSON**, **LaTeX**, and **tabular formats**, making it a valuable tool for documentation and technical data generation.

5. **Multilingual Technical Mastery**
   Delivers consistent results across 20+ languages for math and code—serving global academic and development use cases.

6. **Lightweight Inference-Ready Design**
   Optimized for **edge devices**, **GPUs with limited VRAM**, and **offline deployments**, enabling high-quality results on constrained systems.

---

## **Quickstart with Transformers**

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "prithivMLmods/Nenque-MoT-0.6B-Elite14"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Solve the equation: 2(x - 4) + 3 = 11. Show all steps."

messages = [
    {"role": "system", "content": "You are a step-by-step math tutor."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
```

---

## **Intended Use**

* Step-by-step mathematical reasoning and symbolic computation
* Lightweight multilingual code generation and debugging
* Structured content generation (e.g., LaTeX, JSON, Markdown)
* Academic tutoring and technical assistant roles
* Deployment in resource-constrained or edge scenarios

---

## **Limitations**

* Not suitable for extended creative generation or conversational fluency
* Limited context length impacts performance on long multi-step tasks
* Fine-tuned on technical domains—general chat or abstract logic tasks may underperform
* Specialized for structured outputs—free-form generation is not its focus

---

## **References**

1. [Qwen2.5 Technical Report (2024)](https://arxiv.org/pdf/2412.15115)
2. [YaRN: Efficient Context Window Extension of Large Language Models](https://arxiv.org/pdf/2309.00071)
3. [open-r1/Mixture-of-Thoughts](https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts)