llama_tokenizer_ext_zhtw / added_tokens.json
AlexHung29629's picture
Upload tokenizer
806da72 verified
raw
history blame
67 Bytes
{
"</s>": 2,
"<s>": 1,
"<unk>": 0,
"<|func_end|>": 36128
}