Hugging Face's logo
Michael Goin's picture
82 16 19

Michael Goin PRO

mgoin
srvm's profile picture nickandbro's profile picture kylesayrs's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Recent Activity

new activity 4 days ago
nm-testing/Qwen2.5-VL-72B-Instruct-FP8-Dynamic:Remove image_processor_type
updated a model 4 days ago
nm-testing/Qwen1.5-MoE-A2.7B-Chat-quantized.w4a16
new activity 4 days ago
nm-testing/Qwen2.5-VL-7B-Instruct-FP8-Dynamic:Use Qwen2VLImageProcessor for image_processor_type
View all activity

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Mistral AI_'s profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture Red Hat's profile picture

mgoin's activity

New activity in nm-testing/Qwen2.5-VL-72B-Instruct-FP8-Dynamic 4 days ago

Remove image_processor_type
#2 opened 4 days ago by
pooya-davoodi-parasail
New activity in nm-testing/Qwen2.5-VL-7B-Instruct-FP8-Dynamic 4 days ago

Use Qwen2VLImageProcessor for image_processor_type

5
#2 opened 7 days ago by
pooya-davoodi-parasail

Use Qwen2VLImageProcessor for image_processor_type
#3 opened 5 days ago by
pooya-davoodi-parasail
New activity in cognitivecomputations/DeepSeek-R1-AWQ 15 days ago

when i use vllm v0.7.2 to deploy r1 awq, i got empty content

13
#10 opened 15 days ago by
bupalinyu

MLA is not supported with moe_wna16 quantization. Disabling MLA.

5
#7 opened 17 days ago by
AMOSE
New activity in neuralmagic/gemma-2-9b-it-FP8 23 days ago

AttributeError: 'Gemma2Config' object has no attribute 'interleaved_sliding_window' Traceback (most recent call last):

2
#3 opened 25 days ago by
samos123
New activity in neuralmagic/granite-3.1-8b-instruct-FP8-dynamic 24 days ago

compressed-tensors MLA support requires fp8 activations and weights in group 'group_0',

2
#1 opened 25 days ago by
samos123
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV about 1 month ago

How to load this model?

2
#1 opened 8 months ago by
Frz614
New activity in neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic 2 months ago

Model does not run with VLLM

2
#3 opened 2 months ago by
aswad546
New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic 2 months ago

Nice model, any info on scripts used to quantize?

1
#1 opened 3 months ago by
RonanMcGovern
New activity in neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic 2 months ago

Thanks!
#2 opened 3 months ago by
Jindows
New activity in mistralai/Pixtral-Large-Instruct-2411 3 months ago

Add config_format and load_format to vLLM args
#5 opened 3 months ago by
mgoin

Update config.json to use null for sliding_window
#4 opened 3 months ago by
mgoin
New activity in mgoin/nemotron-3-8b-chat-4k-sft-hf 4 months ago

Adding `safetensors` variant of this model
#1 opened 4 months ago by
SFconvertbot
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 4 months ago

Is this the standard GPTQ quantization?

1
#5 opened 4 months ago by
molereddy
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 4 months ago

Model weights are not loaded

4
#3 opened 6 months ago by
MarvelousMouse
New activity in neuralmagic/pixtral-12b-FP8-dynamic 4 months ago

Update model card
#1 opened 4 months ago by
nm-research
New activity in nm-testing/llava-1.5-7b-hf-FP8-dynamic 4 months ago

Add chat_template to tokenizer_config.json
#1 opened 4 months ago by
nm-research
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 4 months ago

7900xtx torch._scaled_mm is only supported on CUDA devices with compute capability >= 9.0 or 8.9, or ROCm MI300+

1
#3 opened 4 months ago by
aaaaaaaaaasdf
New activity in mistral-community/pixtral-12b 4 months ago

Why is the Pixtral activation function "gelu" when the reference code uses "silu"?

2
#10 opened 5 months ago by
mgoin