I think the question is if the released pre-trained models were trained on a mixture of supervised and unsupervised tasks (so they learned with prefixes and putting them in front of my input when fine-tuning is useful) or only on unsupervised tasks. I know they do it both ways in the paper but not clear which one was released.