How was this fine-tuned?
Fine-tuning was not released. Will it be?
It was fine-tuned like any other LLM: take a small training dataset (in this case the LJSpeech training dataset) and train the model for 10-20 epochs on it.
No it was not released: I with-held the VQVAE model that you would need to (easily) perform fine-tunes.
It was fine-tuned like any other LLM: take a small training dataset (in this case the LJSpeech training dataset) and train the model for 10-20 epochs on it.
No it was not released: I with-held the VQVAE model that you would need to (easily) perform fine-tunes.
Thanks. This is a really unique and interesting project. From tinkering a little I'm finding great promise. Are you planning on releasing the model anytime soon? ✌️ Even privately?
how about the robustness?
I pretrain the tortoise model with 10w hours , and then finetune it with 20 hours dataset of a female voice.
I tried:
1.lr = 0.00001, finetune all the parameters
2.lr = 0.00001, finetune using lora
3.lr = 0.00001, freeze part of the model and only finetune few layers
Thought the synthesized waves sounds good, but the robustness declained.
Repeat sometimes happens especially for long sentences.
For example, the input text is "I think .. blabla ......, I am happy",
the syntheszied wave is "I think .. blabla ......, I am happy, I am happy"
How to finetune to keep robustness?