nan commited on
Commit
69ac66d
·
1 Parent(s): a188cd1

docs: update the transformers and API codes

Browse files
Files changed (1) hide show
  1. README.md +41 -20
README.md CHANGED
@@ -26,13 +26,15 @@ Embeddings produced by `jina-embeddings-v4` serve as the backbone for neural inf
26
 
27
 
28
  Built based on [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct), `jina-embeddings-v4` has the following features:
29
- - **Unified embeddings** for text, images, and documents, supporting both dense (single-vector) and late-interaction (multi-vector) retrieval.
 
30
  - **Multilingual support** (20+ languages) and compatibility with a wide range of domains, including technical and visually complex documents.
31
  - **Task-specific adapters** for retrieval, text matching, and code-related tasks, which can be selected at inference time.
32
  - **Flexible embedding size**: dense embeddings are 2048 dimensions by default but can be truncated to as low as 128 with minimal performance loss.
33
 
34
 
35
  Summary of features:
 
36
  | Feature | Jina Embeddings V4 |
37
  |------------|------------|
38
  | Base Model | Qwen2.5-VL-3B-Instruct |
@@ -42,8 +44,8 @@ Summary of features:
42
  | Single-Vector Dimension | 2048 |
43
  | Multi-Vector Dimension | 128 |
44
  | Matryoshka dimensions | 128, 256, 512, 1024, 2048 |
45
- | Attention Mechanism | FlashAttention2 |
46
  | Pooling Strategy | Mean pooling |
 
47
 
48
 
49
 
@@ -58,6 +60,7 @@ Please refer to our [technical report of jina-embeddings-v4](https://puginarug.c
58
  <summary>Requirements</a></summary>
59
 
60
  The following Python packages are required:
 
61
  - `transformers>=4.52.0`
62
  - `torch>=2.6.0`
63
  - `peft>=0.15.2`
@@ -68,25 +71,21 @@ The following Python packages are required:
68
  - **flash-attention**: Installing [flash-attention](https://github.com/Dao-AILab/flash-attention) is recommended for improved inference speed and efficiency, but not mandatory.
69
  - **sentence-transformers**: If you want to use the model via the `sentence-transformers` interface, install this package as well.
70
 
71
-
72
  </details>
73
 
74
 
75
  <details>
76
- <summary>via Jina AI <a href="https://jina.ai/embeddings/">Embedding API</a></summary>
 
77
 
78
- Needs to be adjusted for V4
79
  ```bash
80
  curl https://api.jina.ai/v1/embeddings \
81
  -H "Content-Type: application/json" \
82
- -H "Authorization: Bearer [JINA_AI_API_TOKEN]" \
83
  -d @- <<EOFEOF
84
  {
85
  "model": "jina-embeddings-v4",
86
- "dimensions": 1024,
87
- "task": "retrieval.query",
88
- "normalized": true,
89
- "embedding_type": "float",
90
  "input": [
91
  {
92
  "text": "غروب جميل على الشاطئ"
@@ -136,37 +135,41 @@ EOFEOF
136
 
137
  ```python
138
  # !pip install transformers>=4.52.0 torch>=2.6.0 peft>=0.15.2 torchvision pillow
139
- # !pip install
140
  from transformers import AutoModel
 
141
 
142
  # Initialize the model
143
  model = AutoModel.from_pretrained("jinaai/jina-embeddings-v4", trust_remote_code=True)
 
 
 
144
  # ========================
145
  # 1. Retrieval Task
146
  # ========================
147
  # Configure truncate_dim, max_length (for texts), max_pixels (for images), vector_type, batch_size in the encode function if needed
148
 
149
  # Encode query
150
- query_embedding = model.encode_texts(
151
  texts=["Overview of climate change impacts on coastal cities"],
152
  task="retrieval",
153
  prompt_name="query",
154
- )[0]
155
 
156
  # Encode passage (text)
157
- passage_embedding = model.encode_texts(
158
  texts=[
159
  "Climate change has led to rising sea levels, increased frequency of extreme weather events..."
160
  ],
161
  task="retrieval",
162
  prompt_name="passage",
163
- )[0]
164
 
165
  # Encode image/document
166
- image_embedding = model.encode_images(
167
  images=["https://i.ibb.co/nQNGqL0/beach1.jpg"],
168
  task="retrieval",
169
- )[0]
170
 
171
  # ========================
172
  # 2. Text Matching Task
@@ -183,25 +186,43 @@ texts = [
183
  "해변 위로 아름다운 일몰", # Korean
184
  ]
185
 
186
- text_embeddings = model.encode_texts(texts=texts, task="text-matching")
187
 
188
  # ========================
189
  # 3. Code Understanding Task
190
  # ========================
191
 
192
  # Encode query
193
- query_embedding = model.encode_texts(
194
  texts=["Find a function that prints a greeting message to the console"],
195
  task="code",
196
  prompt_name="query",
197
  )
198
 
199
  # Encode code
200
- code_embeddings = model.encode_texts(
201
  texts=["def hello_world():\n print('Hello, World!')"],
202
  task="code",
203
  prompt_name="passage",
204
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
205
  ```
206
  </details>
207
 
 
26
 
27
 
28
  Built based on [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct), `jina-embeddings-v4` has the following features:
29
+
30
+ - **Unified embeddings** for text, images, and visual documents, supporting both dense (single-vector) and late-interaction (multi-vector) retrieval.
31
  - **Multilingual support** (20+ languages) and compatibility with a wide range of domains, including technical and visually complex documents.
32
  - **Task-specific adapters** for retrieval, text matching, and code-related tasks, which can be selected at inference time.
33
  - **Flexible embedding size**: dense embeddings are 2048 dimensions by default but can be truncated to as low as 128 with minimal performance loss.
34
 
35
 
36
  Summary of features:
37
+
38
  | Feature | Jina Embeddings V4 |
39
  |------------|------------|
40
  | Base Model | Qwen2.5-VL-3B-Instruct |
 
44
  | Single-Vector Dimension | 2048 |
45
  | Multi-Vector Dimension | 128 |
46
  | Matryoshka dimensions | 128, 256, 512, 1024, 2048 |
 
47
  | Pooling Strategy | Mean pooling |
48
+ | Attention Mechanism | FlashAttention2 |
49
 
50
 
51
 
 
60
  <summary>Requirements</a></summary>
61
 
62
  The following Python packages are required:
63
+
64
  - `transformers>=4.52.0`
65
  - `torch>=2.6.0`
66
  - `peft>=0.15.2`
 
71
  - **flash-attention**: Installing [flash-attention](https://github.com/Dao-AILab/flash-attention) is recommended for improved inference speed and efficiency, but not mandatory.
72
  - **sentence-transformers**: If you want to use the model via the `sentence-transformers` interface, install this package as well.
73
 
 
74
  </details>
75
 
76
 
77
  <details>
78
+ <summary>via <a href="https://jina.ai/embeddings/">Jina AI Embeddings API</a></summary>
79
+
80
 
 
81
  ```bash
82
  curl https://api.jina.ai/v1/embeddings \
83
  -H "Content-Type: application/json" \
84
+ -H "Authorization: Bearer $JINA_AI_API_TOKEN" \
85
  -d @- <<EOFEOF
86
  {
87
  "model": "jina-embeddings-v4",
88
+ "task": "text-matching",
 
 
 
89
  "input": [
90
  {
91
  "text": "غروب جميل على الشاطئ"
 
135
 
136
  ```python
137
  # !pip install transformers>=4.52.0 torch>=2.6.0 peft>=0.15.2 torchvision pillow
138
+ # !pip install
139
  from transformers import AutoModel
140
+ import torch
141
 
142
  # Initialize the model
143
  model = AutoModel.from_pretrained("jinaai/jina-embeddings-v4", trust_remote_code=True)
144
+
145
+ model.to("cuda")
146
+
147
  # ========================
148
  # 1. Retrieval Task
149
  # ========================
150
  # Configure truncate_dim, max_length (for texts), max_pixels (for images), vector_type, batch_size in the encode function if needed
151
 
152
  # Encode query
153
+ query_embeddings = model.encode_text(
154
  texts=["Overview of climate change impacts on coastal cities"],
155
  task="retrieval",
156
  prompt_name="query",
157
+ )
158
 
159
  # Encode passage (text)
160
+ passage_embeddings = model.encode_text(
161
  texts=[
162
  "Climate change has led to rising sea levels, increased frequency of extreme weather events..."
163
  ],
164
  task="retrieval",
165
  prompt_name="passage",
166
+ )
167
 
168
  # Encode image/document
169
+ image_embeddings = model.encode_image(
170
  images=["https://i.ibb.co/nQNGqL0/beach1.jpg"],
171
  task="retrieval",
172
+ )
173
 
174
  # ========================
175
  # 2. Text Matching Task
 
186
  "해변 위로 아름다운 일몰", # Korean
187
  ]
188
 
189
+ text_embeddings = model.encode_text(texts=texts, task="text-matching")
190
 
191
  # ========================
192
  # 3. Code Understanding Task
193
  # ========================
194
 
195
  # Encode query
196
+ query_embedding = model.encode_text(
197
  texts=["Find a function that prints a greeting message to the console"],
198
  task="code",
199
  prompt_name="query",
200
  )
201
 
202
  # Encode code
203
+ code_embeddings = model.encode_text(
204
  texts=["def hello_world():\n print('Hello, World!')"],
205
  task="code",
206
  prompt_name="passage",
207
  )
208
+
209
+ # ========================
210
+ # 4. Use multivectors
211
+ # ========================
212
+
213
+ multivector_embeddings = model.encode_text(
214
+ texts=texts,
215
+ task="retrieval",
216
+ prompt_name="query",
217
+ return_multivector=True,
218
+ )
219
+
220
+ images = ["https://i.ibb.co/nQNGqL0/beach1.jpg", "https://i.ibb.co/r5w8hG8/beach2.jpg"]
221
+ multivector_image_embeddings = model.encode_image(
222
+ images=images,
223
+ task="retrieval",
224
+ return_multivector=True,
225
+ )
226
  ```
227
  </details>
228