08 -> 09

README.md

@@ -14,11 +14,11 @@ co2_eq_emissions: 1
 
 # Model Summary
 
-> OLMoE-1B-7B is a Mixture-of-Experts LLM with 1B active and 7B total parameters released in August 2024 (0824). It yields state-of-the-art performance among models with a similar cost (1B) and is competitive with much larger models like Llama2-13B. OLMoE is 100% open-source.
+> OLMoE-1B-7B is a Mixture-of-Experts LLM with 1B active and 7B total parameters released in September 2024 (0924). It yields state-of-the-art performance among models with a similar cost (1B) and is competitive with much larger models like Llama2-13B. OLMoE is 100% open-source.
 
 - Code: https://github.com/allenai/OLMoE
 - Paper:
-- Logs:
+- Logs: https://wandb.ai/ai2-llm/olmoe/reports/OLMoE-1B-7B-0924--Vmlldzo4OTcyMjU3
 
 # Use
 
@@ -31,8 +31,8 @@ import torch
 DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
 
 # Load different ckpts via passing e.g. `revision=step10000-tokens41B`
-model = OlmoeForCausalLM.from_pretrained("OLMoE/OLMoE-1B-7B-0824").to(DEVICE)
-tokenizer = AutoTokenizer.from_pretrained("OLMoE/OLMoE-1B-7B-0824")
+model = OlmoeForCausalLM.from_pretrained("OLMoE/OLMoE-1B-7B-0924").to(DEVICE)
+tokenizer = AutoTokenizer.from_pretrained("OLMoE/OLMoE-1B-7B-0924")
 inputs = tokenizer("Bitcoin is", return_tensors="pt")
 inputs = {k: v.to(DEVICE) for k, v in inputs.items()}
 out = model.generate(**inputs, max_length=64)
@@ -43,13 +43,13 @@ print(tokenizer.decode(out[0]))
 You can list all revisions/branches by installing `huggingface-hub` & running:
 ```python
 from huggingface_hub import list_repo_refs
-out = list_repo_refs("OLMoE/OLMoE-1B-7B-0824")
+out = list_repo_refs("OLMoE/OLMoE-1B-7B-0924")
 branches = [b.name for b in out.branches]
 ```
 
 Important branches:
 - `step1200000-tokens5033B`: Pretraining checkpoint used for annealing. There are a few more checkpoints after this one but we did not use them.
-- `main`: Checkpoint annealed from `step1200000-tokens5033B` for an additional 100B tokens (23,842 steps). We use this checkpoint for our adaptation (https://huggingface.co/OLMoE/OLMoE-1B-7B-0824-SFT & https://huggingface.co/OLMoE/OLMoE-1B-7B-0824-Instruct).
+- `main`: Checkpoint annealed from `step1200000-tokens5033B` for an additional 100B tokens (23,842 steps). We use this checkpoint for our adaptation (https://huggingface.co/OLMoE/OLMoE-1B-7B-0924-SFT & https://huggingface.co/OLMoE/OLMoE-1B-7B-0924-Instruct).
 - `fp32`: FP32 version of `main`. The model weights were stored in FP32 during training but we did not observe any performance drop from casting them to BF16 after training so we upload all weights in BF16. If you want the original FP32 checkpoint for `main` you can use this one. You will find that it yields slightly different results but should perform around the same on benchmarks.
 
 # Citation