Artificial Intelligence, Research, Machine Learning, NLP.
trained on just over a Trillion tokens. You can see the architecture specs below: Source: https://arxiv.org/pdf/2302.13971 These models are trained on fully public data, and they manage to achieve strong performance on a variety of QA and Common Sense Reasoning tasks both in zero and few-shot.