content.raw: Finally finished my extensive **Qwen 3 evaluations** across a range of formats and quantisations, focusing on **MMLU-Pro** (Computer Science).
content.type: text
content.value: Finally finished my extensive **Qwen 3 evaluations** across a range of formats and quantisations, focusing on **MMLU-Pro** (Computer Science).
content.type: new_line
content.type: new_line
content.raw: A few take-aways stood out - especially for those interested in local deployment and performance trade-offs:
content.type: text
content.value: A few take-aways stood out - especially for those interested in local deployment and performance trade-offs:
content.type: new_line
content.type: new_line
content.raw: 1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s.
content.type: text
content.value: 1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s.
content.type: new_line
content.raw: 2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend.
content.type: text
content.value: 2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend.
content.type: new_line
content.raw: 3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s.
content.type: text
content.value: 3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s.
content.type: new_line
content.raw: 4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups.
content.type: text
content.value: 4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups.
content.type: new_line
content.raw: 5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off).
content.type: text
content.value: 5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off).
content.type: new_line
content.type: new_line
content.raw: All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings.
content.type: text
content.value: All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings.
content.type: new_line
content.type: new_line
content.raw: **Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default.
content.type: text
content.value: **Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default.
content.type: new_line
content.type: new_line
content.raw: Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!
content.type: text
content.value: Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!
rawContent: Finally finished my extensive **Qwen 3 evaluations** across a range of formats and quantisations, focusing on **MMLU-Pro** (Computer Science).
  
  A few take-aways stood out - especially for those interested in local deployment and performance trade-offs:
  
  1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s.
  2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend.
  3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s.
  4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups.
  5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off).
  
  All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings.
  
  **Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default.
  
  Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!
reactions.reaction: 🤗
reactions.users: FlipTip John6666 zenosai djuna eaddario akkikiki biodigital2 daniel-ltw joaompalmeiro shimmyshimmer fatihhangulec zzjcool
reactions.reaction: 🔥
reactions.users: ivanfioravanti Safiy-x biodigital2 daniel-ltw DawnC ilintar MLDataScientist ZennyKenny jozik15 shimmyshimmer
reactions.reaction: 👍
reactions.users: kramp Xodla
reactions.reaction: 🚀
reactions.users: BuiDoan
commentators._id: 634c1dba4abe8405758986e2
commentators.avatarUrl: /avatars/23dbe5c16bebce192a26261ec91547ab.svg
commentators.fullname: Felix Fischer
commentators.name: FlipTip
commentators.type: user
commentators._id: 658f24cb35c41262d64af1a2
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/658f24cb35c41262d64af1a2/6FYHADxnJxdFEc297skfm.jpeg
commentators.fullname: Ed Addario
commentators.name: eaddario
commentators.type: user
commentators._id: 6518b24569c938b3d6109591
commentators.avatarUrl: /avatars/28352f32087e3f8140f791dc55e55a82.svg
commentators.fullname: Bio Digital
commentators.name: biodigital2
commentators.type: user