| # gpt-oss-120b-bf16 | |
| ## Model Introduction | |
| This model is the bf16 version converted from [openai/gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b). | |
| ## Usage | |
| You can use this model in [SGLang](https://github.com/sgl-project/sglang) with the following instructions. | |
| ### Installation | |
| ``` | |
| # build from source | |
| git clone https://github.com/sgl-project/sglang | |
| cd sglang | |
| pip3 install pip --upgrade | |
| pip3 install -e "python[all]" | |
| # ROCm 6.3 | |
| pip3 install torch==2.8.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/rocm6.3 | |
| git clone https://github.com/triton-lang/triton | |
| cd python/triton_kernels | |
| pip3 install . | |
| # hopper | |
| pip3 install torch==2.8.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu126 | |
| pip3 install sgl-kernel==0.3.2 | |
| # blackwell cu128 | |
| pip3 install torch==2.8.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu128 | |
| pip3 install https://github.com/sgl-project/whl/releases/download/v0.3.2/sgl_kernel-0.3.2+cu128-cp39-abi3-manylinux2014_x86_64.whl | |
| # blackwell cu129 | |
| pip3 install torch==2.8.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu129 | |
| pip3 install https://github.com/sgl-project/whl/releases/download/v0.3.2/sgl_kernel-0.3.2-cp39-abi3-manylinux2014_x86_64.whl | |
| ``` | |
| ### Launch command | |
| ``` | |
| python3 -m sglang.launch_server --model lmsys/gpt-oss-120b-bf16 --tp 4 | |
| ``` | |
| ### For more details | |
| https://github.com/sgl-project/sglang/issues/8833 |