Covid19 Related Question Answering (Closed book question answering)
In 2020, COVID-19 which is caused by a coronavirus called SARS-CoV-2 took over the world. It touched the lives of many people and caused a lot of hardship for humanity. There are still many questions in regards to COVID-19 and it is often difficult to get the right answers. The aim of this project is to finetune models for closed book question answering. In closed-book QA, we feed the model a question without any context or access to external knowledge and train it to predict the answer. Since the model doesn’t receive any context, the primary way it can learn to answer these questions is based on the “knowledge” it obtained during pre-training [1]  [2].
The main goals of this project are:

Train a model for question answering in regards to COVID-19
Release the top performing models for further research and enhancement
Release all of the preprocessing and postprocessing scripts and findings for future research.


2. Language
The model will be trained in English and python language.

3. Model
Possible model candidate model is pretrained T5-large 2 or Bert variants (e.g. BioBert orBioClinicalBert) with a sequence generation head.

4. Datasets
The following datasets would be used for finetuning the model. Note that the last dataset is optional and the model is evaluated only using Covid-QA.

Covid-QA 4
CDC-QA 5
Optional - Trivia-QA 2



5. Training scripts
We can make use of :

For preprocessing and mixing datasets 3
For T5 training 2


8. (Optional) Reads
The following links can be useful to better understand the project and
what has previously been done.

How Much Knowledge Can You Pack Into the Parameters of a Language Model? 3