readme: ---
  license: cc-by-nc-4.0
  library_name: fairseq
  task: text-to-speech
  tags:
  - fairseq
  - audio
  - text-to-speech
  language: hk
  ---
  ## unit_hifigan_HK_layer12.km2500_frame_TAT-TTS
  
  Hokkien unit HiFiGAN based vocoder from fairseq:
  - Trained with [TAT-TTS](https://sites.google.com/speech.ntut.edu.tw/fsw/home/tat-tts-corpus) data with 4 speakers in Taiwanese Hokkien accent. See [here]( https://research.facebook.com/publications/hokkien-direct-speech-to-speech-translation) 
  for more training details.
  
  ## Usage
  
  ```python
  import json
  import os
  from pathlib import Path
  
  import IPython.display as ipd
  from fairseq import hub_utils
  from fairseq.checkpoint_utils import load_model_ensemble_and_task_from_hf_hub
  from fairseq.models.speech_to_text.hub_interface import S2THubInterface
  from fairseq.models.text_to_speech import CodeHiFiGANVocoder
  from fairseq.models.text_to_speech.hub_interface import VocoderHubInterface
  
  from huggingface_hub import snapshot_download
  import torchaudio
  
  cache_dir = os.getenv("HUGGINGFACE_HUB_CACHE")
  
  # speech synthesis           
  library_name = "fairseq"
  cache_dir = (
      cache_dir or (Path.home() / ".cache" / library_name).as_posix()
  )
  cache_dir = snapshot_download(
      f"facebook/unit_hifigan_HK_layer12.km2500_frame_TAT-TTS", cache_dir=cache_dir, library_name=library_name
  )
  
  x = hub_utils.from_pretrained(
      cache_dir,
      "model.pt",
      ".",
      archive_map=CodeHiFiGANVocoder.hub_models(),
      config_yaml="config.json",
      fp16=False,
      is_vocoder=True,
  )
  
  with open(f"{x['args']['data']}/config.json") as f:
      vocoder_cfg = json.load(f)
  assert (
      len(x["args"]["model_path"]) == 1
  ), "Too many vocoder models in the input"
  
  vocoder = CodeHiFiGANVocoder(x["args"]["model_path"][0], vocoder_cfg)
  tts_model = VocoderHubInterface(vocoder_cfg, vocoder)
  
  tts_sample = tts_model.get_model_input(unit)
  wav, sr = tts_model.get_prediction(tts_sample)
  
  ipd.Audio(wav, rate=sr)
  ```