jncasey:

And now I’m looking for the best way to use that trained decoder to generate lyrics continuing from a prompt, without feeding the model a title.


Ok makes sense! In that case, suppose that you have already fine-tuned a BART model, then you can access its decoder as follows:
model = BartForConditionalGeneration.from_pretrained('facebook/bart-base')
decoder = model.model.decoder