WordWeaver-AI / added_tokens.json
Zen1t's picture
Upload tokenizer
653bef5 verified
{
"<pad>": 32000,
"[PAD]": 32001
}