RedHatAI
/

Qwen3-32B-FP8-dynamic

Text Generation

compressed-tensors

Model card Files Files and versions Community

alexmarques commited on 5 days ago

Commit

6a33e61

·

verified ·

1 Parent(s): c830130

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -128,7 +128,7 @@ The model was evaluated on the OpenLLM leaderboard tasks (version 1), using [lm-
   ```
   lm_eval \
     --model vllm \
-    --model_args pretrained="RedHatAI/Qwen3-32B-FP8-dynamic",dtype=auto,gpu_memory_utilization=0.5,max_model_len=8192,enable_chunk_prefill=True,tensor_parallel_size=2 \
     --tasks openllm \
     --apply_chat_template\
     --fewshot_as_multiturn \

   ```
   lm_eval \
     --model vllm \
+    --model_args pretrained="RedHatAI/Qwen3-32B-FP8-dynamic",dtype=auto,gpu_memory_utilization=0.5,max_model_len=8192,enable_chunk_prefill=True,tensor_parallel_size=1 \
     --tasks openllm \
     --apply_chat_template\
     --fewshot_as_multiturn \