Artificial Intelligence, Machine Learning.
can be adapted to a wide range of downstream tasks. They were trained through a process of self-supervised learning, processing vast quantities of unlabelled data. Prior to 2020 the largest foundation model, GPT-2(the GPT, in this case, stands for Generative Pre-trained Transformer), was trained on