Hugging Face's logo
Robert Shaw's picture
17 6 4

Robert Shaw

robertgshaw2
joennlae's profile picture brwoodside's profile picture shubhrapandit's profile picture
·

AI & ML interests

None yet

Recent Activity

new activity 6 days ago
neuralmagic/DeepSeek-R1-Distill-Llama-70B-FP8-dynamic:Update tokenizer_config.json
new activity about 2 months ago
nm-testing/pixtral-12b-w4a16-actorder-group:What is an actorder group and what are the advantages of running this in vLLM?
new activity 3 months ago
neuralmagic/Sparse-Llama-3.1-8B-2of4:Can I apply a LoRA?
View all activity

Organizations

HeavyAI's profile picture Neural Magic's profile picture Revel Labs's profile picture NM Testing's profile picture Red Hat's profile picture

robertgshaw2's activity

New activity in neuralmagic/DeepSeek-R1-Distill-Llama-70B-FP8-dynamic 6 days ago

Update tokenizer_config.json

5
#3 opened 6 days ago by
erichartford
New activity in nm-testing/pixtral-12b-w4a16-actorder-group about 2 months ago

What is an actorder group and what are the advantages of running this in vLLM?

4
#1 opened about 2 months ago by
nickandbro
New activity in neuralmagic/Sparse-Llama-3.1-8B-2of4 3 months ago

Can I apply a LoRA?

2
#1 opened 3 months ago by
RonanMcGovern
New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic 3 months ago

Nice model, any info on scripts used to quantize?

1
#1 opened 3 months ago by
RonanMcGovern
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 4 months ago

How to download the model with transformer library

5
#6 opened 4 months ago by
Rick10
New activity in mistralai/Pixtral-12B-2409 4 months ago

Update README.md

3
#25 opened 4 months ago by
robertgshaw2
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 5 months ago

Issue running on vLLM using FP8

2
#3 opened 5 months ago by
ffleandro
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 6 months ago

vllm says the requested model does not exist

2
#1 opened 6 months ago by
shivams101
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 6 months ago

Storage format differs from other w4a16 models

2
#2 opened 6 months ago by
timdettmers
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 6 months ago

Model weights are not loaded

4
#3 opened 6 months ago by
MarvelousMouse
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 8 months ago

Can not be inferenced with vllm openai server

1
#1 opened 8 months ago by
jjqsdq
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-quantized.w4a16 8 months ago

Code example request with vllm

2
#1 opened 8 months ago by
ShiningJazz
New activity in neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit 8 months ago

4bit quantisation does not reduce vram usage.

1
#2 opened 8 months ago by
fu-man
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 8 months ago

How to run Meta-Llama-3-70B-Instruct-FP8 using several devices?

5
#3 opened 8 months ago by
Fertel
New activity in open-llm-leaderboard/open_llm_leaderboard 8 months ago

Reproduction

2
#792 opened 8 months ago by
robertgshaw2
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 10 months ago

Fails to run with nm-vllm

1
#1 opened 10 months ago by
clintonruairi
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_50 11 months ago

Update chart template
#2 opened 11 months ago by
robertgshaw2

Update chart template
#2 opened 11 months ago by
robertgshaw2