readme: ---
  language: 
  - en
  tags:
  - audio
  - automatic-speech-recognition
  - hf-asr-leaderboard
  widget:
  - example_title: Librispeech sample 1
    src: https://cdn-media.huggingface.co/speech_samples/sample1.flac
  - example_title: Librispeech sample 2
    src: https://cdn-media.huggingface.co/speech_samples/sample2.flac
  model-index:
  - name: whisper-tiny.en
    results:
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: LibriSpeech (clean)
        type: librispeech_asr
        config: clean
        split: test
        args: 
          language: en
      metrics:
      - name: Test WER
        type: wer
        value: 8.4372112320138
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: LibriSpeech (other)
        type: librispeech_asr
        config: other
        split: test
        args: 
          language: en
      metrics:
      - name: Test WER
        type: wer
        value: 14.857607503498355
  pipeline_tag: automatic-speech-recognition
  license: apache-2.0
  ---
  
  # Whisper
  
  Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on 680k hours 
  of labelled data, Whisper models demonstrate a strong ability to generalise to many datasets and domains **without** the need 
  for fine-tuning.
  
  Whisper was proposed in the paper [Robust Speech Recognition via Large-Scale Weak Supervision](https://arxiv.org/abs/2212.04356) 
  by Alec Radford et al. from OpenAI. The original code repository can be found [here](https://github.com/openai/whisper).
  
  **Disclaimer**: Content for this model card has partly been written by the Hugging Face team, and parts of it were 
  copied and pasted from the original model card.
  
  ## Model details
  
  Whisper is a Transformer based encoder-decoder model, also referred to as a _sequence-to-sequence_ model. 
  It was trained on 680k hours of labelled speech data annotated using large-scale weak supervision. 
  
  The models were trained on either English-only data or multilingual data. The English-only models were trained 
  on the task of speech recognition. The multilingual models were trained on both speech recognition and speech 
  translation. For speech recognition, the model predicts transcriptions in the *same* language as the audio. 
  For speech translation, the model predicts transcriptions to a *different* language to the audio.
  
  Whisper checkpoints come in five configurations of varying model sizes.
  The smallest four are trained on either English-only or multilingual data.
  The largest checkpoints are multilingual only. All ten of the pre-trained checkpoints 
  are available on the [Hugging Face Hub](https://huggingface.co/models?search=openai/whisper). The 
  checkpoints are summarised in the following table with links to the models on the Hub:
  
  | Size     | Parameters | English-only                                         | Multilingual                                        |
  |----------|------------|------------------------------------------------------|-----------------------------------------------------|
  | tiny     | 39 M       | [✓](https://huggingface.co/openai/whisper-tiny.en)   | [✓](https://huggingface.co/openai/whisper-tiny)     |
  | base     | 74 M       | [✓](https://huggingface.co/openai/whisper-base.en)   | [✓](https://huggingface.co/openai/whisper-base)     |
  | small    | 244 M      | [✓](https://huggingface.co/openai/whisper-small.en)  | [✓](https://huggingface.co/openai/whisper-small)    |
  | medium   | 769 M      | [✓](https://huggingface.co/openai/whisper-medium.en) | [✓](https://huggingface.co/openai/whisper-medium)   |
  | large    | 1550 M     | x                                                    | [✓](https://huggingface.co/openai/whisper-large)    |
  | large-v2 | 1550 M     | x                                                    | [✓](https://huggingface.co/openai/whisper-large-v2) |
  
  # Usage
  
  This checkpoint is an *English-only* model, meaning it can be used for English speech recognition. Multilingual speech 
  recognition or speech translation is possible through use of a multilingual checkpoint.
  
  To transcribe audio samples, the model has to be used alongside a [`WhisperProcessor`](https://huggingface.co/docs/transformers/model_doc/whisper#transformers.WhisperProcessor).
  
  The `WhisperProcessor` is used to:
  1. Pre-process the audio inputs (converting them to log-Mel spectrograms for the model)
  2. Post-process the model outputs (converting them from tokens to text)
  
  ## Transcription
  
  ```python
  >>> from transformers import WhisperProcessor, WhisperForConditionalGeneration
  >>> from datasets import load_dataset
  
  >>> # load model and processor
  >>> processor = WhisperProcessor.from_pretrained("openai/whisper-tiny.en")
  >>> model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-tiny.en")
  
  >>> # load dummy dataset and read audio files
  >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
  >>> sample = ds[0]["