Availability of OLMo-2 Models in Original Format?
Are there any OLMo-2 models available in their original format after training? The ones uploaded here (stage1-step0-tokens0B, stage1-step300-tokens1B, stage1-step10000-tokens21B, etc. ) follow the Hugging Face format, which I believe can’t be used directly as a training checkpoint when specifying the load_path flag in a training config (e.g. https://github.com/allenai/OLMo/blob/main/configs/official-0425/OLMo2-1B-stage1.yaml when training with https://github.com/allenai/OLMo/blob/main/scripts/train.py), though please correct me if that is actually possible.
I’m ideally looking for a non-sharded format with the files:
config.yaml
model.pt
optim.pt
train.pt
Thanks in advance!
Hey @suzeva , you can find links of checkpoints here: https://github.com/allenai/OLMo/blob/main/configs/official-0425/OLMo-2-0425-1B.csv in the same unsharded format you are looking for.
Thanks so much! That was exactly what I needed