openai

/

gpt-oss-120b

like 2.2k

Follow

OpenAI 14.5k

The vocab file should be downloaded onto your filesystem. https://github.com/openai/harmony/blob/main/src/tiktoken_ext/public_encodings.rs#L57

Is the blob storage URL potentially blocked?

Hello @dkundel-openai ,

As mentioned, I'm trying to run gpt-oss-120b on a machine without internet access.

I can pre-download the model artifacts using huggingface-cli and transfer them to the machine.

However, it seems that the huggingface-cli download does not include a vocabulary file.

Is there any workaround for running the model on an air-gapped system?

@rsullenbLL ,

I was able to get around this by grabbing the tiktoken vocab file on a machine connected to the internet. I was using the vllm/vllm-openai:gptoss Docker image with success on an air-gapped system. Steps below:

1. In a clean Python 3.12 environment, install the openai_harmony package: pip install openai-harmony
2. Run the following command: TIKTOKEN_RS_CACHE_DIR=/vocab/download/path python -c 'from openai_harmony import load_harmony_encoding; load_harmony_encoding("HarmonyGptOss")'
3. In TIKTOKEN_RS_CACHE_DIR, you should see a downloaded file named after its hash. When I was running it the file was named fb374d419588a4632f3f557e76b4b70aebbca790, but not sure how often this changes.
4. Move that file to your offline machine
5. When running, make sure to set TIKTOKEN_RS_CACHE_DIR to wherever you moved that file on the offline machine

It took some digging through the OpenAI Harmony GitHub to find that env variable. See this file/line for reference.
The main thing that took me a while to find out was that env variable. I was used to using TIKTOKEN_CACHE_DIR, but since OpenAI Harmony has some Rust-compiled Python libraries they use TIKTOKEN_RS_CACHE_DIR.

Thank you! TIKTOKEN_CACHE_DIR was tripping me up too.

Setting TIKTOKEN_RS_CACHE_DIR did the trick.

Appreciate your help here, nice guide for reference later.

Is the blob storage URL potentially blocked?

In this URL, https://openaipublic.blob.core.windows.net/encodings/
I didn't find any resources.

I did the exact same steps @mthreet proposes. However, the file fb374d419588a4632f3f557e76b4b70aebbca790 gets deleted all the time after starting vllm and receiving the above error. Any ideas why this might be the case?
Nvm, it works. My upload / download mechanism changed the file (hash haven't matched).