Thanks @nielsr, I’ll check that class out!
And I understand that I’ll get best results using the model as-trained. But I’m messing around with AI in creative projects, so “best” isn’t as important to me as “interesting” at the moment.
In the case of the project that prompted my original question, I had fine-tuned a BART model on song lyrics and song titles – first in the way you’d expect, to generate a title to a given set of lyrics, and then with the columns swapped, to generate lyrics given a title. And now I’m looking for the best way to use that trained decoder to generate lyrics continuing from a prompt, without feeding the model a title.
And this question will also come into play on a future project I’m planning, which needs a new language model for both CLM and seq2seq tasks. If I can get by with just training the language model once, it’ll save me a ton of time.