(after the pooling layer) that goes through a linear layer (to go from number of features at the end of the resnet to
the hidden state dimension of the transformer).
The different inputs are concatenated, and on top of the positional embeddings, a segment embedding is added to let the
model know which part of the input vector corresponds to the text and which to the image.
The pretrained model only works for classification.
More information in this [model documentation](model_doc/mmbt). TODO: write this page
Retrieval-based models
Some models use documents retrieval during (pre)training and inference for open-domain question answering, for example.
DPR
Dense Passage Retrieval for Open-Domain Question Answering, Vladimir Karpukhin et
al.
Dense Passage Retrieval (DPR) - is a set of tools and models for state-of-the-art open-domain question-answering
