tohoku-nlp
/

roberta-base-japanese

Inference Endpoints

Model card Files Files and versions Community

roberta-base-japanese / tokenizer_config.json

kiyono's picture

initial commit

8e0dd0e about 3 years ago

No virus

155 Bytes

	{
	"do_lower_case": false,
	"word_tokenizer_type": "mecab",
	"mecab_kwargs": {
	"mecab_dic": "unidic_lite"
	},
	"do_zenkaku": true
	}