text: Title: [Bug] Streaming inference does not work
  Body: ### Describe the bug
  
  Tried the streaming code at https://docs.coqui.ai/en/latest/models/xtts.html#streaming-manually
  with use_deepspeed=False on CPU. Got error: AttributeError: 'int' object has no attribute '_pad_token_tensor'
  
  
  
  ### To Reproduce
  
  import os
  import time
  import torch
  import torchaudio
  from TTS.tts.configs.xtts_config import XttsConfig
  from TTS.tts.models.xtts import Xtts
  
  print("Loading model...")
  config = XttsConfig()
  config.load_json("/path/to/xtts/config.json")
  model = Xtts.init_from_config(config)
  model.load_checkpoint(config, checkpoint_dir="/path/to/xtts/", use_deepspeed=False)
  #model.cuda()
  
  print("Computing speaker latents...")
  gpt_cond_latent, speaker_embedding = model.get_conditioning_latents(audio_path=["reference.wav"])
  
  print("Inference...")
  t0 = time.time()
  chunks = model.inference_stream(
      "It took me quite a long time to develop a voice and now that I have it I am not going to be silent.",
      "en",
      gpt_cond_latent,
      speaker_embedding
  )
  
  wav_chuncks = []
  for i, chunk in enumerate(chunks):
      if i == 0:
          print(f"Time to first chunck: {time.time() - t0}")
      print(f"Received chunk {i} of audio length {chunk.shape[-1]}")
      wav_chuncks.append(chunk)
  wav = torch.cat(wav_chuncks, dim=0)
  torchaudio.save("xtts_streaming.wav", wav.squeeze().unsqueeze(0).cpu(), 24000)
  
  ### Expected behavior
  
  It should output generated audio
  
  ### Logs
  
  ```shell
  /Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/TTS/tts/layers/xtts/stream_generator.py:138: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)
    warnings.warn(
  /Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:818: UserWarning: `return_dict_in_generate` is NOT set to `True`, but `output_hidden_states` is. When `return_dict_in_generate` is not `True`, `output_hidden_states` is ignored.
    warnings.warn(
  Traceback (most recent call last):
    File "/Users/zhz/Desktop/paradigm/conversation_playground/xtts/xtts_streaming.py", line 31, in <module>
      for i, chunk in enumerate(chunks):
    File "/Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 36, in generator_context
      response = gen.send(None)
    File "/Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/TTS/tts/models/xtts.py", line 652, in inference_stream
      gpt_generator = self.gpt.get_generator(
    File "/Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/TTS/tts/layers/xtts/gpt.py", line 603, in get_generator
      return self.gpt_inference.generate_stream(
    File "/Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
      return func(*args, **kwargs)
    File "/Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/TTS/tts/layers/xtts/stream_generator.py", line 186, in generate
      model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation(
    File "/Users/zhz/miniconda3/envs/xtts/lib/python3.10/site-packages/transformers/generation/utils.py", line 585, in _prepare_attention_mask_for_generation
      pad_token_id = generation_config._pad_token_tensor
  AttributeError: 'int' object has no attribute '_pad_token_tensor'
  ```
  
  
  ### Environment
  
  ```shell
  '{
      "CUDA": {
          "GPU": [],
          "available": false,
          "version": null
      },
      "Packages": {
          "PyTorch_debug": false,
          "PyTorch_version": "2.5.1",
          "TTS": "0.22.0",
          "numpy": "1.22.0"
      },
      "System": {
          "OS": "Darwin",
          "architecture": [
              "64bit",
              ""
          ],
          "processor": "arm",
          "python": "3.10.16",
          "version": "Darwin Kernel Version 24.1.0: Thu Oct 10 21:06:23 PDT 2024; root:xnu-11215.41.3~3/RELEASE_ARM64_T8132"
      }
  }
  ```
  
  
  ### Additional context
  
  _No response_