DynaMoE: Dynamic Token-Level Expert Activation with Layer-Wise Adaptive Capacity for Mixture-of-Experts Neural Networks
Paper
• 2603.01697 • Published
• 1
AI & ML interests
Exploring Extreme Quantization techniques !
Recent Activity
Goekdeniz-Guelmez
authored a paper about 5 hours ago
Parveshiiii
posted an
update 20 days ago
Post
275
Introducing Seekify — a truly non‑rate‑limiting search library for Python
Tired of hitting rate limits when building search features? I’ve built Seekify, a lightweight Python library that lets you perform searches without the usual throttling headaches.
🔹 Key highlights
- Simple API — plug it in and start searching instantly
- No rate‑limiting restrictions
- Designed for developers who need reliable search in projects, scripts, or apps
📦 Available now on PyPI:
👉 Check out the repo: https:/github.com/Parveshiiii/Seekify
I’d love feedback, contributions, and ideas for real‑world use cases. Let’s make search smoother together!
Tired of hitting rate limits when building search features? I’ve built Seekify, a lightweight Python library that lets you perform searches without the usual throttling headaches.
🔹 Key highlights
- Simple API — plug it in and start searching instantly
- No rate‑limiting restrictions
- Designed for developers who need reliable search in projects, scripts, or apps
📦 Available now on PyPI:
pip install seekify👉 Check out the repo: https:/github.com/Parveshiiii/Seekify
I’d love feedback, contributions, and ideas for real‑world use cases. Let’s make search smoother together!
Parveshiiii
posted an
update about 1 month ago
Post
1615
🚀 Wanna train your own AI Model or Tokenizer from scratch?
Building models isn’t just for big labs anymore — with the right data, compute, and workflow, you can create **custom AI models** and **tokenizers** tailored to any domain. Whether it’s NLP, domain‑specific datasets, or experimental architectures, training from scratch gives you full control over vocabulary, embeddings, and performance.
✨ Why train your own?
- Full control over vocabulary & tokenization
- Domain‑specific optimization (medical, legal, technical, etc.)
- Better performance on niche datasets
- Freedom to experiment with architectures
⚡ The best part?
- Tokenizer training (TikToken / BPE) can be done in **just 3 lines of code**.
- Model training runs smoothly on **Google Colab notebooks** — no expensive hardware required.
📂 Try out my work:
- 🔗 https://github.com/OE-Void/Tokenizer-from_scratch
- 🔗 https://github.com/OE-Void/GPT
Building models isn’t just for big labs anymore — with the right data, compute, and workflow, you can create **custom AI models** and **tokenizers** tailored to any domain. Whether it’s NLP, domain‑specific datasets, or experimental architectures, training from scratch gives you full control over vocabulary, embeddings, and performance.
✨ Why train your own?
- Full control over vocabulary & tokenization
- Domain‑specific optimization (medical, legal, technical, etc.)
- Better performance on niche datasets
- Freedom to experiment with architectures
⚡ The best part?
- Tokenizer training (TikToken / BPE) can be done in **just 3 lines of code**.
- Model training runs smoothly on **Google Colab notebooks** — no expensive hardware required.
📂 Try out my work:
- 🔗 https://github.com/OE-Void/Tokenizer-from_scratch
- 🔗 https://github.com/OE-Void/GPT
Parveshiiii
posted an
update about 1 month ago
Post
243
📢 The Announcement
Subject: XenArcAI is now Modotte – A New Chapter Begins! 🚀
Hello everyone,
We are thrilled to announce that XenArcAI is officially rebranding to Modotte!
Since our journey began, we’ve been committed to pushing the boundaries of AI through open-source innovation, research, and high-quality datasets. As we continue to evolve, we wanted a name that better represents our vision for a modern, interconnected future in the tech space.
What is changing?
The Name: Moving forward, all our projects, models, and community interactions will happen under the Modotte banner.
The Look: You’ll see our new logo and a fresh color palette appearing across our platforms.
What is staying the same?
The Core Team: It’s still the same people behind the scenes, including our founder, Parvesh Rawal.
Our Mission: We remain dedicated to releasing state-of-the-art open-source models and datasets.
Our Continuity: All existing models, datasets, and projects will remain exactly as they are—just with a new home.
This isn’t just a change in appearance; it’s a commitment to our next chapter of growth and discovery. We are so grateful for your ongoing support as we step into this new era.
Welcome to the future. Welcome to Modotte.
Best regards, The Modotte Team
Subject: XenArcAI is now Modotte – A New Chapter Begins! 🚀
Hello everyone,
We are thrilled to announce that XenArcAI is officially rebranding to Modotte!
Since our journey began, we’ve been committed to pushing the boundaries of AI through open-source innovation, research, and high-quality datasets. As we continue to evolve, we wanted a name that better represents our vision for a modern, interconnected future in the tech space.
What is changing?
The Name: Moving forward, all our projects, models, and community interactions will happen under the Modotte banner.
The Look: You’ll see our new logo and a fresh color palette appearing across our platforms.
What is staying the same?
The Core Team: It’s still the same people behind the scenes, including our founder, Parvesh Rawal.
Our Mission: We remain dedicated to releasing state-of-the-art open-source models and datasets.
Our Continuity: All existing models, datasets, and projects will remain exactly as they are—just with a new home.
This isn’t just a change in appearance; it’s a commitment to our next chapter of growth and discovery. We are so grateful for your ongoing support as we step into this new era.
Welcome to the future. Welcome to Modotte.
Best regards, The Modotte Team
Goekdeniz-Guelmez
authored a paper 2 months ago
Parveshiiii
posted an
update 2 months ago
Post
3581
Hey everyone!
We’re excited to introduce our new Telegram group: https://t.me/XenArcAI
This space is built for **model builders, tech enthusiasts, and developers** who want to learn, share, and grow together. Whether you’re just starting out or already deep into AI/ML, you’ll find a supportive community ready to help with knowledge, ideas, and collaboration.
💡 Join us to:
- Connect with fellow developers and AI enthusiasts
- Share your projects, insights, and questions
- Learn from others and contribute to a growing knowledge base
👉 If you’re interested, hop in and be part of the conversation: https://t.me/XenArcAI
We’re excited to introduce our new Telegram group: https://t.me/XenArcAI
This space is built for **model builders, tech enthusiasts, and developers** who want to learn, share, and grow together. Whether you’re just starting out or already deep into AI/ML, you’ll find a supportive community ready to help with knowledge, ideas, and collaboration.
💡 Join us to:
- Connect with fellow developers and AI enthusiasts
- Share your projects, insights, and questions
- Learn from others and contribute to a growing knowledge base
👉 If you’re interested, hop in and be part of the conversation: https://t.me/XenArcAI
-
- 12 replies
Parveshiiii
posted an
update 4 months ago
Post
1659
Another banger from XenArcAI! 🔥
We’re thrilled to unveil three powerful new releases that push the boundaries of AI research and development:
🔗 https://huggingface.co/XenArcAI/SparkEmbedding-300m
- A lightning-fast embedding model built for scale.
- Optimized for semantic search, clustering, and representation learning.
🔗 https://huggingface.co/datasets/XenArcAI/CodeX-7M-Non-Thinking
- A massive dataset of 7 million code samples.
- Designed for training models on raw coding patterns without reasoning layers.
🔗 https://huggingface.co/datasets/XenArcAI/CodeX-2M-Thinking
- A curated dataset of 2 million code samples.
- Focused on reasoning-driven coding tasks, enabling smarter AI coding assistants.
Together, these projects represent a leap forward in building smarter, faster, and more capable AI systems.
💡 Innovation meets dedication.
🌍 Knowledge meets responsibility.
We’re thrilled to unveil three powerful new releases that push the boundaries of AI research and development:
🔗 https://huggingface.co/XenArcAI/SparkEmbedding-300m
- A lightning-fast embedding model built for scale.
- Optimized for semantic search, clustering, and representation learning.
🔗 https://huggingface.co/datasets/XenArcAI/CodeX-7M-Non-Thinking
- A massive dataset of 7 million code samples.
- Designed for training models on raw coding patterns without reasoning layers.
🔗 https://huggingface.co/datasets/XenArcAI/CodeX-2M-Thinking
- A curated dataset of 2 million code samples.
- Focused on reasoning-driven coding tasks, enabling smarter AI coding assistants.
Together, these projects represent a leap forward in building smarter, faster, and more capable AI systems.
💡 Innovation meets dedication.
🌍 Knowledge meets responsibility.
Parveshiiii
posted an
update 4 months ago
Post
3057
SparkEmbedding - SoTA cross lingual retrieval
Iam very happy to announce our latest embedding model sparkembedding-300m base on embeddinggemma-300m we fine tuned it on 1m extra examples spanning over 119 languages and result is this model achieves exceptional cross lingual retrieval
Model: https://huggingface.co/XenArcAI/SparkEmbedding-300m
Iam very happy to announce our latest embedding model sparkembedding-300m base on embeddinggemma-300m we fine tuned it on 1m extra examples spanning over 119 languages and result is this model achieves exceptional cross lingual retrieval
Model: https://huggingface.co/XenArcAI/SparkEmbedding-300m
Parveshiiii
posted an
update 5 months ago
Post
221
AIRealNet - SoTA - Image detection model
We’re proud to release AIRealNet — a binary image classifier built to detect whether an image is AI-generated or a real human photograph. Based on SwinV2 and fine-tuned on the AI-vs-Real dataset, this model is optimized for high-accuracy classification across diverse visual domains.
If you care about synthetic media detection or want to explore the frontier of AI vs human realism, we’d love your support. Please like the model and try it out. Every download helps us improve and expand future versions.
Model page: https://huggingface.co/XenArcAI/AIRealNet
We’re proud to release AIRealNet — a binary image classifier built to detect whether an image is AI-generated or a real human photograph. Based on SwinV2 and fine-tuned on the AI-vs-Real dataset, this model is optimized for high-accuracy classification across diverse visual domains.
If you care about synthetic media detection or want to explore the frontier of AI vs human realism, we’d love your support. Please like the model and try it out. Every download helps us improve and expand future versions.
Model page: https://huggingface.co/XenArcAI/AIRealNet
Parveshiiii
posted an
update 5 months ago
Post
4503
Ever wanted an open‑source deep research agent? Meet Deepresearch‑Agent 🔍🤖
1. Multi‑step reasoning: Reflects between steps, fills gaps, iterates until evidence is solid.
2. Research‑augmented: Generates queries, searches, synthesizes, and cites sources.
3. Fullstack + LLM‑friendly: React/Tailwind frontend, LangGraph/FastAPI backend; works with OpenAI/Gemini.
🔗 GitHub: https://github.com/Parveshiiii/Deepresearch-Agent
1. Multi‑step reasoning: Reflects between steps, fills gaps, iterates until evidence is solid.
2. Research‑augmented: Generates queries, searches, synthesizes, and cites sources.
3. Fullstack + LLM‑friendly: React/Tailwind frontend, LangGraph/FastAPI backend; works with OpenAI/Gemini.
🔗 GitHub: https://github.com/Parveshiiii/Deepresearch-Agent
Parveshiiii
posted an
update 5 months ago
Post
3116
🚀 Big news from XenArcAI!
We’ve just released our new dataset: **Bhagwat‑Gita‑Infinity** 🌸📖
✨ What’s inside:
- Verse‑aligned Sanskrit, Hindi, and English
- Clean, structured, and ready for ML/AI projects
- Perfect for research, education, and open‑source exploration
🔗 Hugging Face: https://huggingface.co/datasets/XenArcAI/Bhagwat-Gita-Infinity
Let’s bring timeless wisdom into modern AI together 🙌
We’ve just released our new dataset: **Bhagwat‑Gita‑Infinity** 🌸📖
✨ What’s inside:
- Verse‑aligned Sanskrit, Hindi, and English
- Clean, structured, and ready for ML/AI projects
- Perfect for research, education, and open‑source exploration
🔗 Hugging Face: https://huggingface.co/datasets/XenArcAI/Bhagwat-Gita-Infinity
Let’s bring timeless wisdom into modern AI together 🙌
Parveshiiii
posted an
update 5 months ago
Post
2466
🚀 New Release from XenArcAI
We’re excited to introduce AIRealNet — our SwinV2‑based image classifier built to distinguish between artificial and real images.
✨ Highlights:
- Backbone: SwinV2
- Input size: 256×256
- Labels: artificial vs. real
- Performance: Accuracy 0.999 | F1 0.999 | Val Loss 0.0063
This model is now live on Hugging Face:
👉 https://huggingface.co/XenArcAI/AIRealNet
We built AIRealNet to push forward open‑source tools for authenticity detection, and we can’t wait to see how the community uses it.
We’re excited to introduce AIRealNet — our SwinV2‑based image classifier built to distinguish between artificial and real images.
✨ Highlights:
- Backbone: SwinV2
- Input size: 256×256
- Labels: artificial vs. real
- Performance: Accuracy 0.999 | F1 0.999 | Val Loss 0.0063
This model is now live on Hugging Face:
👉 https://huggingface.co/XenArcAI/AIRealNet
We built AIRealNet to push forward open‑source tools for authenticity detection, and we can’t wait to see how the community uses it.
Post
3287
🚀 Ever dreamed of training your own Large Language Model from scratch? What if I told you it doesn't require a supercomputer or PhD in ML? 🤯
Introducing LLM Trainer - the educational framework that makes LLM training accessible to EVERYONE! Whether you're on a CPU-only laptop or scaling to distributed GPUs, we've got you covered. 💻➡️🖥️
Why LLM Trainer? Because existing tools are either too simplistic (hiding the magic) or too complex (requiring expert knowledge). We bridge the gap with:
🎓 Educational transparency - every component built from scratch with clear code
💻 CPU-first approach - start training immediately, no GPU needed
🔧 Full customization - modify anything you want
📈 Seamless scaling - from laptop to cluster without code changes
🤝 HuggingFace integration - works with existing models & tokenizers
Key highlights:
✅ Built-in tokenizers (BPE, WordPiece, HF wrappers)
✅ Complete Transformer implementation from scratch
✅ Optimized for CPU training
✅ Advanced features: mixed precision, gradient checkpointing, multiple generation strategies
✅ Comprehensive monitoring & metrics
Perfect for:
- Students learning transformers
- Researchers prototyping new ideas
- Developers building domain-specific models
Ready to train your first LLM? It's easier than you think!
🔗 Check it out: https://github.com/HelpingAI/llm-trainer
📚 Docs: Getting Started Guide
💬 Join the community: GitHub Discussions
#AI #MachineLearning #LLM #DeepLearning #OpenSource #Python #HuggingFace #NLP
Special thanks to HuggingFace and PyTorch teams for the amazing ecosystem! 🙏
Introducing LLM Trainer - the educational framework that makes LLM training accessible to EVERYONE! Whether you're on a CPU-only laptop or scaling to distributed GPUs, we've got you covered. 💻➡️🖥️
Why LLM Trainer? Because existing tools are either too simplistic (hiding the magic) or too complex (requiring expert knowledge). We bridge the gap with:
🎓 Educational transparency - every component built from scratch with clear code
💻 CPU-first approach - start training immediately, no GPU needed
🔧 Full customization - modify anything you want
📈 Seamless scaling - from laptop to cluster without code changes
🤝 HuggingFace integration - works with existing models & tokenizers
Key highlights:
✅ Built-in tokenizers (BPE, WordPiece, HF wrappers)
✅ Complete Transformer implementation from scratch
✅ Optimized for CPU training
✅ Advanced features: mixed precision, gradient checkpointing, multiple generation strategies
✅ Comprehensive monitoring & metrics
Perfect for:
- Students learning transformers
- Researchers prototyping new ideas
- Developers building domain-specific models
Ready to train your first LLM? It's easier than you think!
🔗 Check it out: https://github.com/HelpingAI/llm-trainer
📚 Docs: Getting Started Guide
💬 Join the community: GitHub Discussions
#AI #MachineLearning #LLM #DeepLearning #OpenSource #Python #HuggingFace #NLP
Special thanks to HuggingFace and PyTorch teams for the amazing ecosystem! 🙏
- 1 reply
1024m
authored a paper 7 months ago
Parveshiiii
posted an
update 7 months ago
Post
1119
🚀 Just Dropped: MathX-5M — Your Gateway to Math-Savvy GPTs
👨🔬 Wanna fine-tune your own GPT for math?
🧠 Building a reasoning agent that actually *thinks*?
📊 Benchmarking multi-step logic across domains?
Say hello to [**MathX-5M**](https://huggingface.co/datasets/XenArcAI/MathX-5M) — a **5 million+ sample** dataset crafted for training and evaluating math reasoning models at scale.
Built by **XenArcAI**, it’s optimized for:
- 🔍 Step-by-step reasoning with
- 🧮 Coverage from arithmetic to advanced algebra and geometry
- 🧰 Plug-and-play with Gemma, Qwen, Mistral, and other open LLMs
- 🧵 Compatible with Harmony, Alpaca, and OpenChat-style instruction formats
Whether you're prototyping a math tutor, testing agentic workflows, or just want your GPT to solve equations like a pro—**MathX-5M is your launchpad**.
🔗 Dive in: (https://huggingface.co/datasets/XenArcAI/MathX-5M)
Let’s make open-source models *actually* smart at math.
#FineTuneYourGPT #MathX5M #OpenSourceAI #LLM #XenArcAI #Reasoning #Gemma #Qwen #Mistral
👨🔬 Wanna fine-tune your own GPT for math?
🧠 Building a reasoning agent that actually *thinks*?
📊 Benchmarking multi-step logic across domains?
Say hello to [**MathX-5M**](https://huggingface.co/datasets/XenArcAI/MathX-5M) — a **5 million+ sample** dataset crafted for training and evaluating math reasoning models at scale.
Built by **XenArcAI**, it’s optimized for:
- 🔍 Step-by-step reasoning with
- 🧮 Coverage from arithmetic to advanced algebra and geometry
- 🧰 Plug-and-play with Gemma, Qwen, Mistral, and other open LLMs
- 🧵 Compatible with Harmony, Alpaca, and OpenChat-style instruction formats
Whether you're prototyping a math tutor, testing agentic workflows, or just want your GPT to solve equations like a pro—**MathX-5M is your launchpad**.
🔗 Dive in: (https://huggingface.co/datasets/XenArcAI/MathX-5M)
Let’s make open-source models *actually* smart at math.
#FineTuneYourGPT #MathX5M #OpenSourceAI #LLM #XenArcAI #Reasoning #Gemma #Qwen #Mistral
1024m
authored a paper 7 months ago
Post
4170
🚀 Dhanishtha-2.0-preview-0825 Is Here
The Intermediate Thinking Model just leveled up again.
With sharper reasoning, better tool use, and expanded capabilities, Dhanishtha-2.0-preview-0825 is now live and ready to impress.
🧠 What Makes Dhanishtha Special?
Unlike typical CoT models that only thinks one time, Dhanishtha thinks iteratively:
> Think → Answer → Rethink → Improve → Rethink again if needed.
🔗 Try it now: HelpingAI/Dhanishtha-2.0-preview-0825
🔞 Dhanishtha NSFW Preview
For those exploring more expressive and immersive roleplay scenarios, we’re also releasing:
HelpingAI/Dhanishtha-nsfw
A specialized version tuned for adult-themed interactions and character-driven roleplay.
🔗 Explore it here: HelpingAI/Dhanishtha-nsfw
💬 You can also try all of these live at chat.helpingai.co
The Intermediate Thinking Model just leveled up again.
With sharper reasoning, better tool use, and expanded capabilities, Dhanishtha-2.0-preview-0825 is now live and ready to impress.
🧠 What Makes Dhanishtha Special?
Unlike typical CoT models that only thinks one time, Dhanishtha thinks iteratively:
> Think → Answer → Rethink → Improve → Rethink again if needed.
🔗 Try it now: HelpingAI/Dhanishtha-2.0-preview-0825
🔞 Dhanishtha NSFW Preview
For those exploring more expressive and immersive roleplay scenarios, we’re also releasing:
HelpingAI/Dhanishtha-nsfw
A specialized version tuned for adult-themed interactions and character-driven roleplay.
🔗 Explore it here: HelpingAI/Dhanishtha-nsfw
💬 You can also try all of these live at chat.helpingai.co
- 4 replies
Parveshiiii
posted an
update 7 months ago
Post
1102
🚀 Launch Alert: Dev-Stack-Agents
Meet your 50-agent senior AI team — principal-level experts in engineering, AI, DevOps, security, product, and more — all bundled into one modular repo.
🔧 Built for Claude Code
Seamlessly plug into Claude’s dev environment:
* 🧠 Each
* ⚙️ Claude indexes them as agents with roles, skills & strategy
* 🤖 You chat → Claude auto-routes to the right agent(s)
* ✍️ Want precision? Just call
* 👥 Complex task? Mention multiple agents for team execution
Examples:
🔗 https://github.com/Parveshiiii/Dev-Stack-Agents
MIT License | Claude-Ready | PRs Welcome
Meet your 50-agent senior AI team — principal-level experts in engineering, AI, DevOps, security, product, and more — all bundled into one modular repo.
+ Code. Optimize. Scale. Secure.
- Full-stack execution, Claude-powered. No human bottlenecks.🔧 Built for Claude Code
Seamlessly plug into Claude’s dev environment:
* 🧠 Each
.md file = a fully defined expert persona* ⚙️ Claude indexes them as agents with roles, skills & strategy
* 🤖 You chat → Claude auto-routes to the right agent(s)
* ✍️ Want precision? Just call
@agent-name directly* 👥 Complex task? Mention multiple agents for team execution
Examples:
"@security-auditor please review auth flow for risks"
"@cloud-architect + @devops-troubleshooter → design a resilient multi-region setup"
"@ai-engineer + @legal-advisor → build a privacy-safe RAG pipeline"🔗 https://github.com/Parveshiiii/Dev-Stack-Agents
MIT License | Claude-Ready | PRs Welcome
Post
2645
ZML just released a technical preview of their new Inference Engine: LLMD.
- Just 2.4GB container, which means fast startup times and efficient autoscaling
- Cross-Platform GPU Support: works on both NVIDIA and AMD GPUs.
- written in Zig
I just tried it out and deployed it on Hugging Face Inference Endpoints and wrote a quick guide 👇 You can try it in like 5 minutes!
https://huggingface.co/blog/erikkaum/test-driving-llmd-inference-engine
- Just 2.4GB container, which means fast startup times and efficient autoscaling
- Cross-Platform GPU Support: works on both NVIDIA and AMD GPUs.
- written in Zig
I just tried it out and deployed it on Hugging Face Inference Endpoints and wrote a quick guide 👇 You can try it in like 5 minutes!
https://huggingface.co/blog/erikkaum/test-driving-llmd-inference-engine
- 1 reply
Post
2158
We just released native support for @SGLang and @vllm-project in Inference Endpoints 🔥
Inference Endpoints is becoming the central place where you deploy high performance Inference Engines.
And that provides the managed infra for it. Instead of spending weeks configuring infrastructure, managing servers, and debugging deployment issues, you can focus on what matters most: your AI model and your users 🙌
Inference Endpoints is becoming the central place where you deploy high performance Inference Engines.
And that provides the managed infra for it. Instead of spending weeks configuring infrastructure, managing servers, and debugging deployment issues, you can focus on what matters most: your AI model and your users 🙌