Update README.md

README.md

@@ -7,13 +7,19 @@ datasets:
 - allenai/olmo-mix-1124
 ---
 
-# SuperBPE Tokenizer
-SuperBPE extends the BPE algorithm to train tokenizers that include both traditional subword tokens (contained within word boundaries), as well as new **superword** tokens (containing parts of multiple words)! This tokenizer has a vocabulary size of 200k and transitions from learning subword to learning superword tokens at vocabulary size of 180k. It is trained on a random subset of Olmo2's pretraining data.
+# SuperBPE
+This 8B model was trained from scratch with a SuperBPE tokenizer. [SuperBPE](https://arxiv.org/abs/2503.13423) extends the BPE algorithm to include both traditional subword tokens (contained within word boundaries), as well as new **superword** tokens (containing parts of multiple words)! Due to encoding the same amount of text in fewer tokens, this model is **27% more efficient at inference-time** on average compared to a model trained with BPE.
+
+The model was trained with the Olmo2 7B architecture and pretraining data. It has a context length of 3,000 tokens (to match the effective context size in bytes of a BPE model with a context length of 4,096 tokens), and is trained on 331B tokens. The tokenizer has a vocabulary size of 200k and transitions from learning subword to learning superword tokens at vocabulary size of 180k.
 
 ## Example Usage
+
 ```
-from transformers import AutoTokenizer
-tokenizer = AutoTokenizer.from_pretrained("alisawuffles/OLMo2-8B-SuperBPE-t180k")
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("UW/OLMo2-8B-SuperBPE-t180k")
+model = AutoModelForCausalLM.from_pretrained("UW/OLMo2-8B-SuperBPE-t180k")
+
 tokenizer.convert_ids_to_tokens(tokenizer.encode("By the way, I am a fan of the Milky Way."))
 # ['ByĠtheĠway', ',ĠIĠam', 'Ġa', 'Ġfan', 'ĠofĠthe', 'ĠMilkyĠWay', '.']
 ```