readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  model-index:
  - name: wav2vec2-large-xls-r-300m-german-with-lm
    results: []
  ---
  
  # wav2vec2-large-xls-r-300m-german-with-lm
  
  This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the German set of the Common Voice dataset.
  It achieves a Word Error Rate of 8,8 percent on the evaluation set
  
  ## Model description
  
  German wav2vec2-xls-r-300m trained on the full train set of Common Voice dataset with a n-gram language model.
  Full code available in [my Github repository](https://github.com/MichaelFleck92/asr-wav2vec)
  
  ## Citation
  
  Feel free to cite this work by
  ```
  @misc{mfleck/wav2vec2-large-xls-r-300m-german-with-lm,
    title={XLS-R-300 Wav2Vec2 German with language model},
    author={Fleck, Michael},
    publisher={Hugging Face},
    journal={Hugging Face Hub},
    howpublished={\url{https://huggingface.co/mfleck/wav2vec2-large-xls-r-300m-german-with-lm}},
    year={2022}
  }
  ```
  
  ## Intended uses & limitations
  
  Inference Usage
  ```python
  from transformers import pipeline
  
  pipe = pipeline(model="mfleck/wav2vec2-large-xls-r-300m-german-with-lm")
  output = pipe("/path/to/file.wav",chunk_length_s=5, stride_length_s=1)
  print(output["text"])
  ```
  
  ## Training and evaluation data
  
  Script used for training (takes about 80 hours on a single A100 40GB)
  
  ```python
  import random
  import re
  import json
  from typing import Any, Dict, List, Optional, Union
  
  import pandas as pd
  import numpy as np
  import torch
  # import soundfile
  
  from datasets import load_dataset, load_metric, Audio
  from dataclasses import dataclass, field
  
  from transformers import Wav2Vec2CTCTokenizer, Wav2Vec2FeatureExtractor, Wav2Vec2Processor, TrainingArguments, Trainer, Wav2Vec2ForCTC
  
  
  '''
      Most parts of this script are following the tutorial: https://huggingface.co/blog/fine-tune-xlsr-wav2vec2
  '''
  
  
  common_voice_train = load_dataset("common_voice", "de", split="train+validation")
  # Use train dataset with less training data
  #common_voice_train = load_dataset("common_voice", "de", split="train[:3%]")
  common_voice_test = load_dataset("common_voice", "de", split="test")
  
  
  # Remove unused columns
  common_voice_train = common_voice_train.remove_columns(["accent", "age", "client_id", "down_votes", "gender", "locale", "segment", "up_votes"])
  common_voice_test = common_voice_test.remove_columns(["accent", "age", "client_id", "down_votes", "gender", "locale", "segment", "up_votes"])
  
  
  # Remove batches with chars which do not exist in German
  print(len(common_voice_train))
  regex = "[^A-Za-zäöüÄÖÜß,?.! ]+"
  common_voice_train = common_voice_train.filter(lambda example: bool(re.search(regex, example['sentence']))==False)
  common_voice_test = common_voice_test.filter(lambda example: bool(re.search(regex, example['sentence']))==False)
  print(len(common_voice_train))
  
  
  # Remove special chars from transcripts
  chars_to_remove_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�\']'
  def remove_special_characters(batch):
      batch["sentence"] = re.sub(chars_to_remove_regex, '', batch["sentence"]).lower()
      return batch
  common_voice_train = common_voice_train.map(remove_special_characters, num_proc=10)
  common_voice_test = common_voice_test.map(remove_special_characters, num_proc=10)
  
  
  # Show some random transcripts to proof that preprocessing worked as expected
  def show_random_elements(dataset, num_examples=10):
      assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
      picks = []
      for _ in range(num_examples):
          pick = random.randint(0, len(dataset)-1)
          while pick in picks:
              pick = random.randint(0, len(dataset)-1)
          picks.append(pick)
      
      print(str(dataset[picks]))
  show_random_elements(common_voice_train.remove_columns(["path","audio"]))
  
  
  # Extract all chars which exist in datasets and add wav2vek tokens
  def extract_all_chars(batch):
      all_text = " ".join(batch["sentence"])
      vocab = list(set(all_text))
      return {"vocab": [vocab], "all_text": [all_text]}
  vocab_train = common_voice_train.map(extract_all_chars, batched=True, batch_size=-1, keep_in_memory=True, remove_columns=common_voice_train.column_names)
  vocab_test = common_voice_test.map(extract_all_chars, batched=True, batch_size=-1, keep_in_memory=True, remove_columns=common_voice_test.column_names)
  
  vocab_list = list(set(vocab_train["vocab"][0]) | set(vocab_test["vocab"][0]))
  vocab_dict = {v: k for k, v in enumerate(sorted(vocab_list))}
  vocab_dict
  vocab_dict["|"] = vocab_dict[" "]
  del vocab_dict[" "]
  vocab_dict["[UNK]"] = len(vocab_dict)
  vocab_dict["[PAD]"] = len(vocab_dict)
  len(vocab_dict)
  with open('vocab.json', 'w') as vocab_file:
      json.dump(vocab_dict, vocab_file)
  
  
  
  # Create tokenizer and repo at Huggingface
  tokenizer = Wav2Vec2CTCTokenizer.from_pretrained("./", unk_token="[UNK]", pad_token="[PAD]", word_delimiter_token="|")
  repo_name = "wav2vec2-large-xls-r-300m-german-with-lm"
  tokenizer.push_to_hub(repo_name)
  print("pushed to h