text: Title: [Bug] In 0.22.0 load_checkpoint  speaker_file_path = speaker_file_path or os.path.join(checkpoint_dir, "speakers_xtts.pth")
  Body: ### Describe the bug
  
  When performing this after fine tuning a model:
  
  def load_model(xtts_checkpoint, xtts_config, xtts_vocab):
      global XTTS_MODEL
      clear_gpu_cache()
      if not xtts_checkpoint or not xtts_config or not xtts_vocab:
          return "You need to run the previous steps or manually set the `XTTS checkpoint path`, `XTTS config path`, and `XTTS vocab path` fields !!"
      config = XttsConfig()
      config.load_json(xtts_config)
      XTTS_MODEL = Xtts.init_from_config(config)
      print("[FINETUNE] \033[94mStarting Step 3\033[0m Loading XTTS model!")
      XTTS_MODEL.load_checkpoint(config, checkpoint_path=xtts_checkpoint, vocab_path=xtts_vocab, use_deepspeed=False)
      if torch.cuda.is_available():
          XTTS_MODEL.cuda()
  
      print("[FINETUNE] Model Loaded!")
      return "Model Loaded!"
      
     TTS version 0.21.3 works absolutely perfectly. 
     
     TTS version 0.22.0 errors in the following way:
  
         XTTS_MODEL.load_checkpoint(config, checkpoint_path=xtts_checkpoint, vocab_path=xtts_vocab, use_deepspeed=False)
    File "C:\AI\text-generation-webui\installer_files\env\Lib\site-packages\TTS\tts\models\xtts.py", line 759, in load_checkpoint
      speaker_file_path = speaker_file_path or os.path.join(checkpoint_dir, "speakers_xtts.pth")
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<frozen ntpath>", line 108, in join
  TypeError: expected str, bytes or os.PathLike object, not NoneType
     
     
  
  ### To Reproduce
  
  I'm using finetuning and loading a model.
  
  ### Expected behavior
  
  to load the model.
  
  ### Logs
  
  ```shell
  XTTS_MODEL.load_checkpoint(config, checkpoint_path=xtts_checkpoint, vocab_path=xtts_vocab, use_deepspeed=False)
    File "C:\AI\text-generation-webui\installer_files\env\Lib\site-packages\TTS\tts\models\xtts.py", line 759, in load_checkpoint
      speaker_file_path = speaker_file_path or os.path.join(checkpoint_dir, "speakers_xtts.pth")
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<frozen ntpath>", line 108, in join
  TypeError: expected str, bytes or os.PathLike object, not NoneType
  ```
  
  
  ### Environment
  
  ```shell
  INFO:root:OS Version: Windows 10.0.22621
  INFO:root:Note: Windows 11 build is 10.x.22xxx
  INFO:root:Python Version: 3.11.5
  INFO:root:Torch Version: 2.1.2+cu121
  INFO:root:System RAM: 22.37 GB available out of 31.34 GB total
  INFO:root:CUDA_HOME: C:\AI\text-generation-webui\installer_files\env
  INFO:root:Package Versions:
  INFO:root:numpy               Required: >= 1.22.0        Installed: 1.24.4
  INFO:root:soundfile           Required: >= 0.12.1        Installed: 0.12.1
  INFO:root:uvicorn             Required: >= 0.24.0.post1  Installed: 0.25.0
  INFO:root:TTS                 Required: >= 0.22.0        Installed: 0.22.0
  INFO:root:torch               Required: >= 2.1.0+cu118   Installed: 2.1.2+cu121
  INFO:root:torchaudio          Required: >= 2.1.0+cu118   Installed: 2.1.2+cu121
  INFO:root:fastapi             Required: >= 0.104.1       Installed: 0.105.0
  INFO:root:Jinja2              Required: >= 3.1.2         Installed: 3.1.2
  INFO:root:requests            Required: >= 2.31.0        Installed: 2.31.0
  INFO:root:tqdm                Required: >= 4.66.1        Installed: 4.66.1
  INFO:root:importlib-metadata  Required: >= 4.8.1         Installed: 7.0.1
  INFO:root:packaging           Required: >= 23.2          Installed: 23.2
  INFO:root:GPU Information:
  Sun Dec 24 11:21:21 2023       
  +---------------------------------------------------------------------------------------+
  | NVIDIA-SMI 546.17                 Driver Version: 546.17       CUDA Version: 12.3     |
  |-----------------------------------------+----------------------+----------------------+
  | GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
  |                                         |                      |               MIG M. |
  |=========================================+======================+======================|
  |   0  NVIDIA GeForce RTX 4070      WDDM  | 00000000:01:00.0  On |                  N/A |
  |  0%   37C    P8              11W / 200W |    597MiB / 12282MiB |      1%      Default |
  |                                         |                      |                  N/A |
  +-----------------------------------------+----------------------+----------------------+
                                                                                           
  +---------------------------------------------------------------------------------------+
  | Processes:                                                                            |
  |  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
  |        ID   ID                                                             Usage      |
  |=======================================================================================|
  |    0   N/A  N/A      1308    C+G   C:\Windows\explorer.exe                   N/A      |
  |    0   N/A  N/A      1952    C+G   ...crosoft\Edge\Application\msedge.exe    N/A      |
  |    0   N/A  N/A      2900    C+G   ...__8wekyb3d8bbwe\Notepad\Notepad.exe    N/A      |
  |    0   N/A  N/A      7528    C+G   ...siveControlPanel\SystemSettings.exe    N/A      |
  |    0   N/A  N/A      7624    C+G   ...5n1h2txyewy\ShellExperienceHost.exe    N/A      |
  |    0   N/A  N/A      7628    C+G   C:\Windows\explorer.exe                   N/A      |
  |    0   N/A  N/A      8840    C+G   ...nr4m\radeonsoftware\AMDRSSrcExt.exe    N/A      |
  |    0   N/A  N/A      9364    C+G   ...nt.CBS_cw5n1h2txyewy\SearchHost.exe    N/A      |
  |    0   N/A  N/A      9412    C+G   ...2txyewy\StartMenuExperienceHost.exe    N/A      |
  |    0   N/A  N/A      9964    C+G   ...__8wekyb3d8bbwe\WindowsTerminal.exe    N/A      |
  |    0   N/A  N/A     12324    C+G   ...m\radeonsoftware\RadeonSoftware.exe    N/A      |
  |    0   N/A  N/A     14164    C+G   ...\cef\cef.win7x64\steamwebhelper.exe    N/A      |
  |    0   N/A  N/A     14640    C+G   ...CBS_cw5n1h2txyewy\TextInputHost.exe    N/A      |
  |    0   N/A  N/A     17072    C+G   ...ft Office\root\Office16\OUTLOOK.EXE    N/A      |
  |    0   N/A  N/A     17420    C+G   ...s (x86)\Razer\Synapse\RzSynapse.exe    N/A      |
  |    0   N/A  N/A     19616    C+G   ...2.0_x64__cv1g1gvanyjgm\WhatsApp.exe    N/A      |
  |    0   N/A  N/A     20624    C+G   ...on\120.0.2210.91\msedgewebview2.exe    N/A      |
  |    0   N/A  N/A     23484    C+G   ...Programs\Microsoft VS Code\Code.exe    N/A      |
  +---------------------------------------------------------------------------------------+
  
  INFO:root:Package Versions:
  INFO:root:absl-py>= 2.0.0
  INFO:root:accelerate>= 0.25.0
  INFO:root:aiofiles>= 23.2.1
  INFO:root:aiohttp>= 3.9.1
  INFO:root:aiosignal>= 1.3.1
  INFO:root:altair>= 5.2.0
  INFO:root:annotated-types>= 0.6.0
  INFO:root:antlr4-python3-runtime>= 4.9.3
  INFO:root:anyascii>= 0.3.2
  INFO:root:anyio>= 3.7.1
  INFO:root:appdirs>= 1.4.4
  INFO:root:asttokens>= 2.4.1
  INFO:root:attributedict>= 0.3.0
  INFO:root:attrs>= 23.1.0
  INFO:root:audioread>= 3.0.1
  INFO:root:autoawq>= 0.1.7
  INFO:root:auto-gptq>= 0.6.0+cu121
  INFO:root:av>= 10.0.0
  INFO:root:Babel>= 2.14.0
  INFO:root:bangla>= 0.0.2
  INFO:root:beautifulsoup4>= 4.12.2
  INFO:root:bitsandbytes>= 0.41.1
  INFO:root:blessings>= 1.7
  INFO:root:blinker>= 1.7.0
  INFO:root:blis>= 0.7.11
  INFO:root:bnnumerizer>= 0.0.2
  INFO:root:bnunicodenormalizer>= 0.1.6
  INFO:root:cachetools>= 5.3.2
  INFO:root:catalogue>= 2.0.10
  INFO:root:certifi>= 2022.12.7
  INFO:root:cffi>= 1.16.0
  INFO:root:chardet>= 5.2.0
  INFO:root:charset-normalizer>= 2.1.1
  INFO:root:click>= 8.1.7
  INFO:root:cloudpathlib>= 0.16.0
  INFO:root:codecov>= 2.1.13
  INFO:root:colorama>= 0.4.6
  INFO:root:coloredlogs>= 15.0.1
  INFO:root:colour-runner>= 0.1.1
  INFO:root:confection>= 0.1.4
  INFO:root:contourpy>= 1.2.0
  INFO:root:coqpit>= 0.0.17
  INFO:root:coverage>= 7.3.4
  INFO:root:cramjam>= 2.7.0
  INFO:root:ctransformers>= 0.2.27+cu121
  INFO:root:ctranslate2>= 3.23.0
  INFO:root:cycler>= 0.12.1
  INFO:root:cymem>= 2.0.8
  INFO:root:Cython>= 3.0.7
  INFO:root:DataProperty>= 1.0.1
  INFO:root:datasets>= 2.16.0
  INFO:root:dateparser>= 1.1.8
  INFO:root:decorator>= 5.1.1
  INFO:root:deepdiff>= 6.7.1
  INFO:root:deep-translator>= 1.9.2
  INFO:root:dill>= 0.3.7
  INFO:root:diskcache>= 5.6.3
  INFO:root:distlib>= 0.3.8
  INFO:root:docker-pycreds>= 0.4.0
  INFO:root:docopt>= 0.6.2
  INFO:root:einops>= 0.7.0
  INFO:root:encodec>= 0.1.1
  INFO:root:evaluate>= 0.4.1
  INFO:root:executing>= 2.0.1
  INFO:root:exllama>= 0.0.18+cu121
  INFO:root:exllamav2>= 0.0.11+cu121
  INFO:root:fastapi>= 0.105.0
  INFO:root:faster-whisper>= 0.10.0
  INFO:root:fastparquet>= 2023.10.1
  INFO:root:ffmpeg>= 1.4
  INFO:root:ffmpy>= 0.3.1
  INFO:root:filelock>= 3.13.1
  INFO:root:flash-attn>= 2.3.4
  INFO:root:Flask>= 3.0.0
  INFO:root:flask-cloudflared>= 0.0.14
  INFO:root:flatbuffers>= 23.5.26
  INFO:root:fonttools>= 4.47.0
  INFO:root:frozenlist>= 1.4.1
  INFO:root:fsspec>= 2023.10.0
  INFO:root:g2pkk>= 0.1.2
  INFO:root:gekko>= 1.0.6
  INFO:root:gitdb>= 4.0.11
  INFO:root:GitPython>= 3.1.40
  INFO:root:google-auth>= 2.25.2
  INFO:root:google-auth-oauthlib>= 1.2.0
  INFO:root:gptq-for-llama>= 0.1.1+cu121
  INFO:root:gradio>= 3.50.2
  INFO:root:gradio_client>= 0.6.1
  INFO:root:grpcio>= 1.60.0
  INFO:root:gruut>= 2.2.3
  INFO:root:gruut-ipa>= 0.13.0
  INFO:root:gruut-lang-de>= 2.0.0
  INFO:root:gruut-lang-en>= 2.0.0
  INFO:root:gruut-lang-es>= 2.0.0
  INFO:root:gruut-lang-fr>= 2.0.2
  INFO:root:h11>= 0.14.0
  INFO:root:hangul-romanize>= 0.1.0
  INFO:root:hqq>= 0.1.1.post1
  INFO:root:httpcore>= 1.0.2
  INFO:root:httpx>= 0.26.0
  INFO:root:huggingface-hub>= 0.20.1
  INFO:root:humanfriendly>= 10.0
  INFO:root:idna>= 3.4
  INFO:root:importlib-metadata>= 7.0.1
  INFO:root:importlib-resources>= 6.1.1
  INFO:root:inflect>= 7.0.0
  INFO:root:inspecta>= 0.1.3
  INFO:root:ipython>= 8.19.0
  INFO:root:itsdangerous>= 2.1.2
  INFO:root:jamo>= 0.4.1
  INFO:root:jedi>= 0.19.1
  INFO:root:jieba>= 0.42.1
  INFO:root:Jinja2>= 3.1.2
  INFO:root:joblib>= 1.3.2
  INFO:root:jsonlines>= 1.2.0
  INFO:root:jsonschema>= 4.20.0
  INFO:root:jsonschema-specifications>= 2023.11.2
  INFO:root:kiwisolver>= 1.4.5
  INFO:root:langcodes>= 3.3.0
  INFO:root:lazy_loader>= 0.3
  INFO:root:librosa>= 0.10.1
  INFO:root:llama_cpp_python>= 0.2.24+cpuavx2
  INFO:root:llama_cpp_python_cuda>= 0.2.24+cu121
  INFO:root:llama_cpp_python_cuda_tensorcores>= 0.2.24+cu121
  INFO:root:llvmlite>= 0.41.1
  INFO:root:lm_eval>= 0.4.0
  INFO:root:lxml>= 4.9.4
  INFO:root:Markdown>= 3.5.1
  INFO:root:markdown-it-py>= 3.0.0
  INFO:root:MarkupSafe>= 2.1.3
  INFO:root:matplotlib>= 3.8.2
  INFO:root:matplotlib-inline>= 0.1.6
  INFO:root:mbstrdecoder>= 1.1.3
  INFO:root:mdurl>= 0.1.2
  INFO:root:more-itertools>= 10.1.0
  INFO:root:mpmath>= 1.3.0
  INFO:root:msgpack>= 1.0.7
  INFO:root:multidict>= 6.0.4
  INFO:root:multiprocess>= 0.70.15
  INFO:root:murmurhash>= 1.0.10
  INFO:root:networkx>= 2.8.8
  INFO:root:ngrok>= 0.12.1
  INFO:root:ninja>= 1.11.1.1
  INFO:root:nltk>= 3.8.1
  INFO:root:num2words>= 0.5.13
  INFO:root:numba>= 0.58.1
  INFO:root:numexpr>= 2.8.8
  INFO:root:numpy>= 1.24.4
  INFO:root:oauthlib>= 3.2.2
  INFO:root:omegaconf>= 2.3.0
  INFO:root:onnxruntime>= 1.16.3
  INFO:root:openai-whisper>= 20231117
  INFO:root:optimum>= 1.16.1
  INFO:root:ordered-set>= 4.1.0
  INFO:root:orjson>= 3.9.10
  INFO:root:packaging>= 23.2
  INFO:root:pandas>= 1.5.3
  INFO:root:parso>= 0.8.3
  INFO:root:pathvalidate>= 3.2.0
  INFO:root:peft>= 0.7.1
  INFO:root:Pillow>= 10.1.0
  INFO:root:pip>= 23.3.1
  INFO:root:platformdirs>= 4.1.0
  INFO:root:pluggy>= 1.3.0
  INFO:root:pooch>= 1.8.0
  INFO:root:portalocker>= 2.8.2
  INFO:root:preshed>= 3.0.9
  INFO:root:prompt-toolkit>= 3.0.43
  INFO:root:protobuf>= 4.23.4
  INFO:root:psutil>= 5.9.7
  INFO:root:pure-eval>= 0.2.2
  INFO:root:pyarrow>= 14.0.2
  INFO:root:pyarrow-hotfix>= 0.6
  INFO:root:pyasn1>= 0.5.1
  INFO:root:pyasn1-modules>= 0.3.0
  INFO:root:pybind11>= 2.11.1
  INFO:root:pycparser>= 2.21
  INFO:root:pydantic>= 2.5.3
  INFO:root:pydantic_core>= 2.14.6
  INFO:root:pydub>= 0.25.1
  INFO:root:Pygments>= 2.17.2
  INFO:root:pynndescent>= 0.5.11
  INFO:root:pyparsing>= 3.1.1
  INFO:root:pypinyin>= 0.50.0
  INFO:root:pyproject-api>= 1.6.1
  INFO:root:pyreadline3>= 3.4.1
  INFO:root:pysbd>= 0.3.4
  INFO:root:pytablewriter>= 1.2.0
  INFO:root:python-crfsuite>= 0.9.10
  INFO:root:python-dateutil>= 2.8.2
  INFO:root:python-multipart>= 0.0.6
  INFO:root:pytz>= 2023.3.post1
  INFO:root:pywin32>= 306
  INFO:root:PyYAML>= 6.0.1
  INFO:root:py-cpuinfo>= 9.0.0
  INFO:root:referencing>= 0.32.0
  INFO:root:regex>= 2023.12.25
  INFO:root:requests>= 2.31.0
  INFO:root:requests-oauthlib>= 1.3.1
  INFO:root:responses>= 0.18.0
  INFO:root:rich>= 13.7.0
  INFO:root:rootpath>= 0.1.1
  INFO:root:rouge>= 1.0.1
  INFO:root:rouge-score>= 0.1.2
  INFO:root:rpds-py>= 0.15.2
  INFO:root:rsa>= 4.9
  INFO:root:sacrebleu>= 2.4.0
  INFO:root:safetensors>= 0.4.1
  INFO:root:scikit-learn>= 1.3.2
  INFO:root:scipy>= 1.11.4
  INFO:root:semantic-version>= 2.10.0
  INFO:root:sentencepiece>= 0.1.99
  INFO:root:sentry-sdk>= 1.39.1
  INFO:root:setproctitle>= 1.3.3
  INFO:root:setuptools>= 68.2.2
  INFO:root:six>= 1.16.0
  INFO:root:smart-open>= 6.4.0
  INFO:root:smmap>= 5.0.1
  INFO:root:sniffio>= 1.3.0
  INFO:root:soundfile>= 0.12.1
  INFO:root:soupsieve>= 2.5
  INFO:root:soxr>= 0.3.7
  INFO:root:spacy>= 3.7.2
  INFO:root:spacy-legacy>= 3.0.12
  INFO:root:spacy-loggers>= 1.0.5
  INFO:root:SpeechRecognition>= 3.10.0
  INFO:root:sqlitedict>= 2.1.0
  INFO:root:srsly>= 2.4.8
  INFO:root:sse-starlette>= 1.6.5
  INFO:root:stack-data>= 0.6.3
  INFO:root:starlette>= 0.27.0
  INFO:root:SudachiDict-core>= 20230927
  INFO:root:SudachiPy>= 0.6.8
  INFO:root:sympy>= 1.12
  INFO:root:tabledata>= 1.3.3
  INFO:root:tabulate>= 0.9.0
  INFO:root:tcolorpy>= 0.1.4
  INFO:root:tensorboard>= 2.15.1
  INFO:root:tensorboard-data-server>= 0.7.2
  INFO:root:termcolor>= 2.4.0
  INFO:root:texttable>= 1.7.0
  INFO:root:thinc>= 8.2.2
  INFO:root:threadpoolctl>= 3.2.0
  INFO:root:tiktoken>= 0.5.2
  INFO:root:timm>= 0.9.12
  INFO:root:tokenizers>= 0.15.0
  INFO:root:toml>= 0.10.2
  INFO:root:toolz>= 0.12.0
  INFO:root:torch>= 2.1.2+cu121
  INFO:root:torchaudio>= 2.1.2+cu121
  INFO:root:torchvision>= 0.16.2+cu121
  INFO:root:tox>= 4.11.4
  INFO:root:tqdm>= 4.66.1
  INFO:root:tqdm-multiprocess>= 0.0.11
  INFO:root:trainer>= 0.0.36
  INFO:root:traitlets>= 5.14.0
  INFO:root:transformers>= 4.36.2
  INFO:root:TTS>= 0.22.0
  INFO:root:typepy>= 1.3.2
  INFO:root:typer>= 0.9.0
  INFO:root:typing_extensions>= 4.9.0
  INFO:root:tzdata>= 2023.3
  INFO:root:tzlocal>= 5.2
  INFO:root:umap-learn>= 0.5.5
  INFO:root:Unidecode>= 1.3.7
  INFO:root:urllib3>= 1.26.13
  INFO:root:uvicorn>= 0.25.0
  INFO:root:virtualenv>= 20.25.0
  INFO:root:wandb>= 0.16.1
  INFO:root:wasabi>= 1.1.2
  INFO:root:wcwidth>= 0.2.12
  INFO:root:weasel>= 0.3.4
  INFO:root:websockets>= 11.0.3
  INFO:root:Werkzeug>= 3.0.1
  INFO:root:wheel>= 0.41.2
  INFO:root:xxhash>= 3.4.1
  INFO:root:yarl>= 1.9.4
  INFO:root:zipp>= 3.17.0
  INFO:root:zstandard>= 0.22.0
  ```
  
  
  ### Additional context
  
  No problems on TTS 0.21.3 as mentioned. Only on TTS 0.22.0