**tiktoken**
[Byte pair encoding](https://texonom.com/byte-pair-encoding-d5c29c3c7dec44c2a4c530a4669124c1) 


[tiktoken](https://github.com/openai/tiktoken)





> [tiktoken (OpenAI) tokenizer — 🦜🔗 LangChain 0.0.194](https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/tiktoken.html)







