BLIP special additional token self.tokenizer.enc_token_id
### System Info

Hello,

It seems to me that the Salesforce implementation of BLIP uses a special tokem 'ENC' at the beginning of each text sentence, it is added 
https://github.com/salesforce/BLIP/blob/a176f1e9cc5a232d2cc6e21b77d2c7e18ceb3c37/models/blip.py#L190

and used when encoding text conditioned on images e.g. https://github.com/salesforce/BLIP/blob/a176f1e9cc5a232d2cc6e21b77d2c7e18ceb3c37/models/blip.py#L67
or here
https://github.com/salesforce/BLIP/blob/b7bb1eeb6e901044a9eb1016f408ee908b216bc7/models/blip_retrieval.py#L124

Shouldn't we do the same? What is special about that token? 

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

When in conditional mode with images, we should add 
`encoder_input_ids[:,0] = self.tokenizer.enc_token_id`?

cc @ArthurZucker <|||||>Hey! Thanks for submitting this issue. I think @younesbelkada is the most familiar with Blip and should be able to answer whether or not the `enc_token_id` should be added. BLIP seems to be using a `BertTokenizer`, and looking at [this](https://huggingface.co/Salesforce/blip-image-captioning-large/blob/main/tokenizer_config.json) I don't think that we are adding it.  <|||||>Any news here? <|||||>Hi @DianeBouchacourt 
Do you notice any notable qualitative difference when adding that token? When porting the model I got predictions matched with the current approach <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.