Wovchena commited on
Commit
2274fa6
·
verified ·
1 Parent(s): c45209e

Extend eos_token_id

Browse files

Extend `eos_token_id` with pad token because that's the token stored in `processor.tokenizer.eos_token_id`. That also simplifies the sample from the model card from
```py
generate_ids = model.generate(**inputs, eos_token_id=processor.tokenizer.eos_token_id, **generation_args)
```
to
```py
generate_ids = model.generate(**inputs, **generation_args)
```

Files changed (1) hide show
  1. config.json +4 -1
config.json CHANGED
@@ -16,7 +16,10 @@
16
  "use_hd_transform": true,
17
  "with_learnable_separator": true
18
  },
19
- "eos_token_id": 2,
 
 
 
20
  "hidden_act": "silu",
21
  "hidden_size": 3072,
22
  "img_processor": {
 
16
  "use_hd_transform": true,
17
  "with_learnable_separator": true
18
  },
19
+ "eos_token_id": [
20
+ 2,
21
+ 32000,
22
+ ],
23
  "hidden_act": "silu",
24
  "hidden_size": 3072,
25
  "img_processor": {