Deep Learning, Artificial Intelligence, Visual Language, Foundation Models, Vlm.
image encoder, which uses the convolutional embedding layers as described in CvT. Two linear projection layers are added on top of the image encoder and language encoder to match the dimensions of image and language features. The model size is 893M, including the language Transformer with 256M