we are very proud to introduce 
text
we are very proud to introduce 
https://huggingface.co/jinaai/jina-clip-v1
jinaai/jina-clip-v1
model
resource
https://huggingface.co/jinaai/jina-clip-v1
, aka "jina-embeddings-multimodal".
text
, aka "jina-embeddings-multimodal".
new_line
new_line
The OpenAI CLIP 
text
The OpenAI CLIP 
https://huggingface.co/openai/clip-vit-base-patch32
openai/clip-vit-base-patch32
model
resource
https://huggingface.co/openai/clip-vit-base-patch32
 have nice performance to align text and image modality, that user can perform cross-modal text image retrieval or image classification on top of it.  However, due to the training data and recipe, it can not:
text
 have nice performance to align text and image modality, that user can perform cross-modal text image retrieval or image classification on top of it.  However, due to the training data and recipe, it can not:
new_line
new_line
1. model longer sequence of text inputs (77 token constraint).
text
1. model longer sequence of text inputs (77 token constraint).
new_line
2. align text representations (CLIP Text Tower is weak for text search).
text
2. align text representations (CLIP Text Tower is weak for text search).
new_line
new_line
In our latest publication, 
text
In our latest publication, 
Jina CLIP: Your CLIP Model Is Also Your Text Retriever (2405.20204)
https://huggingface.co/papers/2405.20204
2405.20204
paper
resource
https://huggingface.co/papers/2405.20204
 , we proposed a multi-task, multi-objective learning scheme. The produced CLIP model shows:
text
 , we proposed a multi-task, multi-objective learning scheme. The produced CLIP model shows:
new_line
new_line
1. Stronger cross-modal performance against OpenAI sets, 2% and 6% improvement on cross-modal retrieval recall@5.
text
1. Stronger cross-modal performance against OpenAI sets, 2% and 6% improvement on cross-modal retrieval recall@5.
new_line
2. Text tower of the JinaCLIP is a strong text encoder, reach the same performance as 
text
2. Text tower of the JinaCLIP is a strong text encoder, reach the same performance as 
https://huggingface.co/jinaai/jina-embeddings-v2-base-en
jinaai/jina-embeddings-v2-base-en
model
resource
https://huggingface.co/jinaai/jina-embeddings-v2-base-en
, 165% improvement on MTEB[BEIR] recall@5.
text
, 165% improvement on MTEB[BEIR] recall@5.
new_line
3. Image tower of the JinaCLIP also shows strong performance in image-image search (CBIR), 12% recall improvement on Cifar100 test set.
text
3. Image tower of the JinaCLIP also shows strong performance in image-image search (CBIR), 12% recall improvement on Cifar100 test set.
new_line
new_line
If you are working on MuRAG (multimodal-retrieval argumented generation), try it out!
text
If you are working on MuRAG (multimodal-retrieval argumented generation), try it out!
new_line
new_line
new_line
we are very proud to introduce https://huggingface.co/jinaai/jina-clip-v1, aka "jina-embeddings-multimodal".

The OpenAI CLIP https://huggingface.co/openai/clip-vit-base-patch32 have nice performance to align text and image modality, that user can perform cross-modal text image retrieval or image classification on top of it.  However, due to the training data and recipe, it can not:

1. model longer sequence of text inputs (77 token constraint).
2. align text representations (CLIP Text Tower is weak for text search).

In our latest publication, https://huggingface.co/papers/2405.20204 , we proposed a multi-task, multi-objective learning scheme. The produced CLIP model shows:

1. Stronger cross-modal performance against OpenAI sets, 2% and 6% improvement on cross-modal retrieval recall@5.
2. Text tower of the JinaCLIP is a strong text encoder, reach the same performance as https://huggingface.co/jinaai/jina-embeddings-v2-base-en, 165% improvement on MTEB[BEIR] recall@5.
3. Image tower of the JinaCLIP also shows strong performance in image-image search (CBIR), 12% recall improvement on Cifar100 test set.

If you are working on MuRAG (multimodal-retrieval argumented generation), try it out!



🚀
florianhoenicke florian-hoenicke zhenwang23 mwerk numb3r3 alaeddineabdessalem osanseviero alexcg2 nan den0620 capricareloaded Jofthomas
❤️
florianhoenicke florian-hoenicke zhenwang23 alaeddineabdessalem osanseviero alexcg2 nan capricareloaded Jofthomas
🔥
florianhoenicke florian-hoenicke alaeddineabdessalem osanseviero alexcg2 nan capricareloaded jaisanrobert
👍
zhenwang23 alaeddineabdessalem osanseviero alexcg2 sbarman25 nan capricareloaded florianhoenicke