nvidia

/

DeepSeek-R1-FP4

like 232

Follow

NVIDIA 21.1k

No

@wang35 Please try this build for A800 (80G) x 8, which enables the FP4 support on A100/A800:

huggingface-cli download nvidia/DeepSeek-R1-FP4 --local-dir ./nvidia/DeepSeek-R1-FP4

docker run -it --rm --ipc=host --privileged -p 8000:8000 \
        --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 --gpus=all -v /:/host -w /host$(pwd) \
        tutelgroup/deepseek-671b:a100x8-chat-20250401 --model_path ./nvidia/DeepSeek-R1-FP4 \
        --prompt "Given f(x) = f(x + f(x)), calculate f(x)."