InterwebAlchemy
/

PGNTokenizer

Model card Files Files and versions Community

Commit History

Revert back to NFD tokenizer

3341d84
verified

InterwebAlchemy commited on Feb 11

Remove normalizer

767e594
verified

InterwebAlchemy commited on Feb 11

Switch from NFD -> NFC

0dcea10
verified

InterwebAlchemy commited on Feb 11

Update README.md

6d5f592
verified

InterwebAlchemy commited on Jan 28

Upload tokenizer

57dcf89
verified

InterwebAlchemy commited on Jan 28

initial commit

74fae32
verified

InterwebAlchemy commited on Jan 25