Qwen3-4B-Instruct-2507-dwq3-mlx

21194/21194 [59:59]

arc_challenge
acc 0.403, norm 0.412, stderr 0.014
arc_easy
acc 0.647, norm 0.539, stderr 0.010
boolq
acc 0.794, norm 0.794, stderr 0.007
hellaswag
acc 0.406, norm 0.475, stderr 0.004
openbookqa
acc 0.292, norm 0.390, stderr 0.021
piqa
acc 0.692, norm 0.686, stderr 0.010
winogrande
acc 0.566, norm 0.566, stderr 0.013

Performance evaluation of the parent model at BF16

21194/21194 [1:02:51]

arc_challenge
acc 0.437, norm 0.441, stderr 0.014
arc_easy
acc 0.711, norm 0.588, stderr 0.010
boolq
acc 0.844, norm 0.844, stderr 0.006
hellaswag
acc 0.391, norm 0.451, stderr 0.004
openbookqa
acc 0.278, norm 0.396, stderr 0.021
piqa
acc 0.701, norm 0.693, stderr 0.010
winogrande
acc 0.558, norm 0.558, stderr 0.013

This model Qwen3-4B-Instruct-2507-dwq3-mlx was converted to MLX format from Qwen/Qwen3-4B-Instruct-2507 using mlx-lm version 0.26.3.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("Qwen3-4B-Instruct-2507-dwq3-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)