more).
The library provides versions of the model for language modeling and multitask language modeling/multiple choice
classification.
CTRL
CTRL: A Conditional Transformer Language Model for Controllable Generation,
Nitish Shirish Keskar et al.
Same as the GPT model but adds the idea of control codes. Text is generated from a prompt (can be empty) and one (or
several) of those control codes which are then used to influence the text generation: generate with the style of
wikipedia article, a book or a movie review.
The library provides a version of the model for language modeling only.
Transformer-XL
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context, Zihang
Dai et al.
Same as a regular GPT model, but introduces a recurrence mechanism for two consecutive segments (similar to a regular
