Hello,
Is the multiple-choice head for the pre-trained LongformerForMultipleChoice model pre-trained as well?
I am asking because like for the pre-trained GPT2DoubleHeadsModel, the main body of the model is trained but its multiple-choice head is not.
Is the multiple-choice head for the pre-trained LongformerForMultipleChoice model untrained, just like the case for the GPT2DoubleHeadsModel?
Thank you,