readme: ---
  language: mt
  datasets:
  - common_voice
  tags:
  - audio
  - automatic-speech-recognition
  - maltese
  - xlrs-53-maltese
  - masri-project
  - malta
  - university-of-malta
  license: cc-by-4.0
  widget:
  model-index:
  - name: wav2vec2-large-xlsr-53-maltese-64h
    results:
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Mozilla Common Voice 11.0 (Test)
        type: mozilla-foundation/common_voice_11_0
        split: test
        args: 
          language: mt
      metrics:
      - name: WER
        type: wer
        value: 1.57
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: Mozilla Common Voice 11.0 (Dev)
        type: mozilla-foundation/common_voice_11_0
        split: validation
        args: 
          language: mt
      metrics:
      - name: WER
        type: wer
        value: 1.40
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: MASRI-TEST Corpus
        type: MLRS/masri_test
        split: test
        args: 
          language: mt
      metrics:
      - name: WER
        type: wer
        value: 27.27
    - task:
        name: Automatic Speech Recognition
        type: automatic-speech-recognition
      dataset:
        name: MASRI-DEV Corpus
        type: MLRS/masri_dev
        split: validation
        args: 
          language: mt
      metrics:
      - name: WER
        type: wer
        value: 24.71
  ---
  
  # wav2vec2-large-xlsr-53-maltese-64h
  
  The "wav2vec2-large-xlsr-53-maltese-64h" is an acoustic model suitable for Automatic Speech Recognition in Maltese. It is the result of fine-tuning the model "facebook/wav2vec2-large-xlsr-53" with around 64 hours of Maltese data developed by the MASRI Project at the University of Malta between 2019 and 2021. Most of the data is available at the the MASRI Project homepage https://www.um.edu.mt/projects/masri/.
  
  The specific list of corpora used to fine-tune the model is:
  
  - MASRI-HEADSET v2 (6h39m)
  - MASRI-Farfield (9h37m)
  - MASRI-Booths (2h27m)
  - MASRI-MEP (1h17m)
  - MASRI-COMVO (7h29m)
  - MASRI-TUBE (13h17m)
  - MASRI-MERLIN (25h18m) *Not available at the MASRI Project homepage
  	
  The fine-tuning process was perform during November (2022) in the servers of the Language and Voice Lab (https://lvl.ru.is/) at Reykjavík University (Iceland) by Carlos Daniel Hernández Mena.
  
  # Evaluation
  
  ```python
  import torch
  from transformers import Wav2Vec2Processor
  from transformers import Wav2Vec2ForCTC
  
  #Load the processor and model.
  MODEL_NAME="carlosdanielhernandezmena/wav2vec2-large-xlsr-53-maltese-64h"
  processor = Wav2Vec2Processor.from_pretrained(MODEL_NAME)
  model = Wav2Vec2ForCTC.from_pretrained(MODEL_NAME)
  
  #Load the dataset
  from datasets import load_dataset, load_metric, Audio
  ds=load_dataset("common_voice", "mt", split="test")
  
  #Normalize the transcriptions
  import re
  chars_to_ignore_regex = '[\\,\\?\\.\\!\\\;\\:\\"\\“\\%\\‘\\”\\�\\)\\(\\*)]'
  def remove_special_characters(batch):
  	batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
  	return batch
  ds = ds.map(remove_special_characters)
  
  #Downsample to 16kHz
  ds = ds.cast_column("audio", Audio(sampling_rate=16_000))
  
  #Process the dataset
  def prepare_dataset(batch):
  	audio = batch["audio"]
  	#Batched output is "un-batched" to ensure mapping is correct
  	batch["input_values"] = processor(audio["array"], sampling_rate=audio["sampling_rate"]).input_values[0]
  	with processor.as_target_processor():
  		batch["labels"] = processor(batch["sentence"]).input_ids
  	return batch
  ds = ds.map(prepare_dataset, remove_columns=ds.column_names,num_proc=1)
  
  #Define the evaluation metric
  import numpy as np
  wer_metric = load_metric("wer")
  def compute_metrics(pred):
  	pred_logits = pred.predictions
  	pred_ids = np.argmax(pred_logits, axis=-1)
  	pred.label_ids[pred.label_ids == -100] = processor.tokenizer.pad_token_id
  	pred_str = processor.batch_decode(pred_ids)
  	#We do not want to group tokens when computing the metrics
  	label_str = processor.batch_decode(pred.label_ids, group_tokens=False)
  	wer = wer_metric.compute(predictions=pred_str, references=label_str)
  	return {"wer": wer}
  
  #Do the evaluation (with batch_size=1)
  model = model.to(torch.device("cuda"))
  def map_to_result(batch):
  	with torch.no_grad():
  		input_values = torch.tensor(batch["input_values"], device="cuda").unsqueeze(0)
  		logits = model(input_values).logits
  	pred_ids = torch.argmax(logits, dim=-1)
  	batch["pred_str"] = processor.batch_decode(pred_ids)[0]
  	batch["sentence"] = processor.decode(batch["labels"], group_tokens=False)
  	return batch
  results = ds.map(map_to_result,remove_columns=ds.column_names)
  
  #Compute the overall WER now.
  print("Test WER: {:.3f}".format(wer_metric.compute(predictions=results["pred_str"], references=results["sentence"])))
  
  ```
  **Test Result**: 0.011
  
  # BibTeX entry and citation info
  *When publishing results based on these models please refer to:*
  ```bibtex
  @misc{mena2022xlrs53maltese,
        title={Acoustic Model in Ma