WhisperTokenizer for two languages at once.
In the blog https://huggingface.co/blog/fine-tune-whisper I see the code piece.

```tokenizer = WhisperTokenizer.from_pretrained("openai/whisper-small", language="Hindi", task="transcribe")```

What if I want to include both French and Hindi?

Any suggestions here?

Thanks and Regards.
cc @ArthurZucker and @sanchit-gandhi <|||||>Hey @BakingBrains! You'll have to set the prefix tokens each time you switch language. You can do this using the [`.set_prefix_tokens`](https://huggingface.co/docs/transformers/model_doc/whisper#transformers.WhisperTokenizer.set_prefix_tokens) method, e.g. for French:
```python
tokenizer.set_prefix_tokens(language="French", task="transcribe")
# encode French target text to label ids 
batch["labels"] = tokenizer(batch["sentence"]).input_ids
```
Then for Hindi:
```python
tokenizer.set_prefix_tokens(language="Hindi", task="transcribe")
# encode Hindi target text to label ids 
batch["labels"] = tokenizer(batch["sentence"]).input_ids
```<|||||>@sanchit-gandhi Thanks a lot. Can't we use both language at once. Like for example I have an audio file where a person speaks Hindi and in between switches the language to French. How can I use the tokenizer in this case?

Thanks and Regards<|||||>I remember trying with a file that contained both French and English and whisper just transcribe the french part as if it was english. The same happens when you try to transcribe some audio that is in english but force the language code to another language: it will write english phonemes corresponding to the sounds that it hears. 
- Now if you want to have a batch, and in the batch you have different language, you can use the `generate` method and provide a batch of `decoder_input_ids` and set the `forced_tokens` to be None. 
- If you have another language in the middle, I suggest trying with the `retur_timestamp` option, which will split the audio. My best recommendation is to do something similar to what openAI's long audio decoding strategy does: when you reach the end of a timestamp, you re-generate. But this is gonna be slow as you don't know if the next sentence is in english or hindi, you cut the next sentence. <|||||>@ArthurZucker Thanks, I have been thinking the same.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.