NLP, Artificial Intelligence, Machine Learning, Data Science, Transformers.
Later evolved into GPT-2 [2019], a bigger and optimized version of GPT pre-trained on WebText, and GPT-3 [2020], a further bigger and optimized version of GPT-2, pre-trained on Common Crawl. CTRL [2019]: similar to GPT but with control codes for conditional text generation. Transformer-XL [2019]: