comet-gpt2-small-japanese / special_tokens_map.json
tide525's picture
Upload tokenizer
898ef81
raw
history blame
No virus
167 Bytes
{
"additional_special_tokens": [
"xNeed",
"xEffect",
"xIntent",
"xReact"
],
"eos_token": "</s>",
"pad_token": "</s>",
"unk_token": "<unk>"
}