dehanalkautsar commited on
Commit
82a28b1
·
verified ·
1 Parent(s): ff20dd9

upload vocab tokenizers

Browse files
ar/final/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "model_max_length": 512}
ar/final/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
ar/final/vocab_ar_parallel_en_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
ar/valid_vocab_ar_translated.txt ADDED
The diff for this file is too large to render. See raw diff
 
ar/valid_vocab_ar_translated_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
ar/vocab_ar_parallel_en.txt ADDED
The diff for this file is too large to render. See raw diff
 
de/final/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "model_max_length": 512}
de/final/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
de/final/vocab_de_parallel_en_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
de/valid_vocab_de_translated.txt ADDED
The diff for this file is too large to render. See raw diff
 
de/valid_vocab_de_translated_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
de/vocab_de_parallel_en.txt ADDED
The diff for this file is too large to render. See raw diff
 
en/final/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "model_max_length": 512}
en/final/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
en/final/vocab_en_parallel_en_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
en/valid_vocab_en.txt ADDED
The diff for this file is too large to render. See raw diff
 
id/final/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "model_max_length": 512}
id/final/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
id/final/vocab_id_parallel_en_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
id/valid_vocab_id_translated.txt ADDED
The diff for this file is too large to render. See raw diff
 
id/valid_vocab_id_translated_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
id/vocab_id_parallel_en.txt ADDED
The diff for this file is too large to render. See raw diff
 
ja/final/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "model_max_length": 512}
ja/final/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
ja/final/vocab_ja_parallel_en_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
ja/valid_vocab_ja_translated.txt ADDED
The diff for this file is too large to render. See raw diff
 
ja/valid_vocab_ja_translated_postprocessed.txt ADDED
The diff for this file is too large to render. See raw diff
 
ja/vocab_ja_parallel_en.txt ADDED
The diff for this file is too large to render. See raw diff