Upload fast tokenizer, which is the recommended and default for transformers now (#1)
Browse files- Upload fast tokenizer, which is the recommended and default for transformers now (4245ca6a111b18e9d95cc3b1317a483c53927921)
Co-authored-by: Tom Jobbins <[email protected]>
- tokenizer.json +0 -0
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|