Hugging Face's logo
Michael Goin's picture
TFLOPS icon 82.6 TFLOPS
69 15 19

Michael Goin

mgoin
nicoboss's profile picture Parkerlambert123's profile picture charlesdedampierre's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture

mgoin's activity

New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 5 days ago

Is this the standard GPTQ quantization?

1
#5 opened 8 days ago by molereddy
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 5 days ago

Model weights are not loaded

4
#3 opened 2 months ago by MarvelousMouse
New activity in neuralmagic/pixtral-12b-FP8-dynamic 9 days ago

Update model card
#1 opened 9 days ago by nm-research
New activity in nm-testing/llava-1.5-7b-hf-FP8-dynamic 9 days ago

Add chat_template to tokenizer_config.json
#1 opened 9 days ago by nm-research
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 9 days ago

7900xtx torch._scaled_mm is only supported on CUDA devices with compute capability >= 9.0 or 8.9, or ROCm MI300+

1
#3 opened 9 days ago by aaaaaaaaaasdf
New activity in mistral-community/pixtral-12b 22 days ago

Why is the Pixtral activation function "gelu" when the reference code uses "silu"?

2
#10 opened 25 days ago by mgoin

Update tokenizer_config.json with chat_template

2
#11 opened 25 days ago by mgoin
New activity in neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic 23 days ago

Any chance your team is working on a 4-bit Llama-3.2-90B-Vision-Instruct-quantized.w4a16 version?

1
#1 opened about 1 month ago by mrhendrey
New activity in neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic about 1 month ago

Oom with 24g vram

3
#1 opened about 1 month ago by Klopez
New activity in neuralmagic/Phi-3.5-mini-instruct-FP8-KV about 1 month ago

latest vllm docker (v0.6.2) fail to load

2
#1 opened about 1 month ago by choronz333
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16 2 months ago

Issue with loading model

1
#1 opened 2 months ago by xSumukhax
New activity in neuralmagic/DeepSeek-Coder-V2-Instruct-FP8 2 months ago

Can it run on A100/A800 with VLLM?

3
#1 opened 3 months ago by Parkerlambert123
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 3 months ago

weights does not exist when trying to deploy in sagemaker endpoint

1
#1 opened 3 months ago by LorenzoCevolaniAXA
New activity in meta-llama/Llama-3.1-405B-Instruct 3 months ago

8-kv-heads

4
#17 opened 3 months ago by ArthurZ
New activity in meta-llama/Llama-3.1-405B 3 months ago

8-kv-heads

3
#21 opened 3 months ago by ArthurZ
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 3 months ago

run with vllm

8
#4 opened 3 months ago by kuliev-vitaly
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8 3 months ago

Not able to run Model using VLLM

1
#3 opened 3 months ago by Pchaudhary
New activity in neuralmagic/gemma-2-9b-it-FP8 3 months ago

getting issue while loading in llm

1
#1 opened 3 months ago by Abhinav6310
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8 3 months ago

How to fast inference with FP8

1
#2 opened 3 months ago by CCRss
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 3 months ago

Unable to load model onto multiple GPUs

2
#2 opened 3 months ago by bprice9