 [Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) for **Closed Book Question Answering**.  The model was pre-trained using T5's denoising objective on [C4](https://huggingface.co/datasets/c4), subsequently additionally pre-trained using [REALM](https://arxiv.org/pdf/2002.08909.pdf)'s salient span masking objective on [Wikipedia](https://huggingface.co/datasets/wikipedia), and finally fine-tuned on [Trivia QA (TQA)](https://huggingface.co/datasets/trivia_qa).  **Note**: The model was fine-tuned on 100% of the train splits of [Trivia QA (TQA)](https://huggingface.co/datasets/trivia_qa) for 10 steps.  Other community Checkpoints: [here](https://huggingface.co/models?search=ssm)  Paper: [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://arxiv.org/abs/1910.10683.pdf)  Authors: *Adam Roberts, Colin Raffel, Noam Shazeer*    