XLM-ProphetNet's model architecture and pretraining objective is same as ProphetNet, but XLM-ProphetNet was pre-trained
on the cross-lingual dataset XGLUE.
The library provides a pre-trained version of this model for multi-lingual conditional generation and fine-tuned
versions for headline generation and question generation, respectively.
Multimodal models
There is one multimodal model in the library which has not been pretrained in the self-supervised fashion like the
others.
MMBT
Supervised Multimodal Bitransformers for Classifying Images and Text, Douwe Kiela
et al.
A transformers model used in multimodal settings, combining a text and an image to make predictions. The transformer
model takes as inputs the embeddings of the tokenized text and the final activations of a pretrained on images resnet
