32 11 37

Saeed

MLDataScientist

AI & ML interests

None yet

Recent Activity

reacted to wolfram's post with 🔥 21 days ago

Finally finished my extensive **Qwen 3 evaluations** across a range of formats and quantisations, focusing on **MMLU-Pro** (Computer Science). A few take-aways stood out - especially for those interested in local deployment and performance trade-offs: 1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s. 2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend. 3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s. 4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups. 5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off). All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings. **Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default. Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!

new activity 26 days ago

btbtyler09/Qwen3-30B-A3B-gptq-8bit:4-bit

new activity about 1 month ago

unsloth/Qwen3-235B-A22B-GGUF:UD quants missing some files

View all activity

Organizations

None yet

MLDataScientist's activity

New activity in btbtyler09/Qwen3-30B-A3B-gptq-8bit 26 days ago

4-bit

#1 opened 28 days ago by

prashanthellina

New activity in unsloth/Qwen3-235B-A22B-GGUF about 1 month ago

UD quants missing some files

➕ 3

#2 opened about 1 month ago by

MLDataScientist

New activity in unsloth/Qwen3-235B-A22B-128K-GGUF about 1 month ago

UD quants missing some files

👀 1

#2 opened about 1 month ago by

MLDataScientist

New activity in MLDataScientist/Mistral-Large-Instruct-2407-GPTQ-3bit 3 months ago

3bit QPTQ quants for Mistral-Large-Instruct-2411

#1 opened 3 months ago by

dazipe

commented a paper 4 months ago

Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs

Paper • 2501.18585 • Published Jan 30 • 61 •

New activity in numen-tech/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GPTQ-Int4 4 months ago

Is this MLC LLM quantized or GPTQ?

#1 opened 4 months ago by

MLDataScientist

New activity in deepseek-ai/DeepSeek-R1 4 months ago

Hardware requirements?

🚀 8

#19 opened 4 months ago by

JohnnieB

New activity in kaitchup/Mistral-Nemo-Base-2407-AutoRound-GPTQ-asym-4bit 5 months ago

Request for Mistral Large 2 Instruct 2407 3bit with Autoround GPTQ

#1 opened 5 months ago by

MLDataScientist

New activity in ChenMnZ/Mistral-Large-Instruct-2407-EfficientQAT-w2g64-GPTQ 9 months ago

Where GGUF?

#1 opened 10 months ago by

rdtfddgrffdgfdghfghdfujgdhgsf

New activity in MLDataScientist/SlimOrca-Dedup-English-Uzbek 10 months ago

[bot] Conversion to Parquet

#1 opened 10 months ago by

parquet-converter

New activity in allenai/ZebraLogic 10 months ago

The problem with tests.

#1 opened 10 months ago by

ZeroWw

New activity in internlm/internlm2_5-7b-chat 11 months ago

Please, add RULER - a long context benchmark

#7 opened 11 months ago by

MLDataScientist

New activity in bartowski/gemma-2-27b-it-GGUF 11 months ago

Prompt format <bos> not needed?

#3 opened 11 months ago by

eamag

New activity in google/gemma-2-27b-it 11 months ago

Hallucinations, misspellings etc. Something seems broken?

#10 opened 11 months ago by

sam-paech

New activity in MaziyarPanahi/calme-2.3-phi3-4b 12 months ago

What dataset is used for fine-tuning?

#9 opened 12 months ago by

MLDataScientist

New activity in MLDataScientist/SFT-UZ-9k 12 months ago

[bot] Conversion to Parquet

#1 opened about 1 year ago by

parquet-converter

New activity in MLDataScientist/Uzbek_news_dataset 12 months ago

[bot] Conversion to Parquet

#1 opened about 1 year ago by

parquet-converter

New activity in MLDataScientist/oasst2_uzbek about 1 year ago

[bot] Conversion to Parquet

#1 opened about 1 year ago by

parquet-converter

New activity in m-a-p/Matrix about 1 year ago

Actual dataset size?

#4 opened about 1 year ago by

jlzhou

New activity in NousResearch/Hermes-2-Pro-Llama-3-8B about 1 year ago

Axolotl Configs Settings

👀 ➕ 5

#9 opened about 1 year ago by

GokhanAI