How to process ARC dataset with HuggingFace GPT2
Hello,

I am interested in processing ARC dataset with HuggingFace GPT2.
The ARC dataset (http://nlpprogress.com/english/question_answering.html) is a question answering, which contains 7,787 genuine grade-school level, multiple-choice science questions. The dataset also comes with the full corpus of texts extracted from various articles that explains various scientific concepts that can be used to solve these 7,787 multiple choice questions (i.e. this full-corpus is not in the multiple choice format; it's just a series of excerpts from various articles)

I am assuming I'd have to use the GPT2DoubleHeadsModel to process this ARC dataset, since it is a set of multiple-choice questions. However, I also need to somehow train my GPT2DoubleHeadsModel based on the contents of the full corpus of texts that contains excerpts from various scientific articles, since GPT2DoubleHeadsModel wouldn't have acquired any scientific knowledge prior to processing this dataset. 

But the thing is, the corpus of series of scientific articles on which I am interested in training my GPT2DoubleHeadsModel with is not written in a multiple-choice format -- is it possible to train only those parts of the GPT2DoubleHeadsModel that are responsible for language modelling with the series of scientific articles, and then fine-tune the entire component of the GPT2DoubleHeadsModel with the training data from the multiple choice questions? 

If it is possible, how can I do it?

Thank you,
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
