boboliu commited on
Commit
9c14746
·
verified ·
1 Parent(s): e484dc5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -1
README.md CHANGED
@@ -19,7 +19,17 @@ VRAM Usage: `3228M` -> `2124M`
19
 
20
  ## What's the cost?
21
 
22
- I think `~5%` accuracy, further evaluation on the way...
 
 
 
 
 
 
 
 
 
 
23
 
24
  ## How to use it?
25
 
 
19
 
20
  ## What's the cost?
21
 
22
+ ~1.69% lost in C-MTEB.
23
+
24
+ | C-MTEB | Param. | Mean(Task) | Mean(Type) | Class. | Clust. | Pair Class. | Rerank. | Retr. | STS |
25
+ |------------------|--------|------------|------------|--------|--------|-------------|---------|-------|-------|
26
+ | multilingual-e5-large-instruct | 0.6B | 58.08 | 58.24 | 69.80 | 48.23 | 64.52 | 57.45 | 63.65 | 45.81 |
27
+ | bge-multilingual-gemma2 | 9B | 67.64 | 75.31 | 59.30 | 86.67 | 68.28 | 73.73 | 55.19 | - |
28
+ | gte-Qwen2-1.5B-instruct | 1.5B | 67.12 | 67.79 | 72.53 | 54.61 | 79.5 | 68.21 | 71.86 | 60.05 |
29
+ | gte-Qwen2-7B-instruct | 7.6B | 71.62 | 72.19 | 75.77 | 66.06 | 81.16 | 69.24 | 75.70 | 65.20 |
30
+ | ritrieve_zh_v1 | 0.3B | 72.71 | 73.85 | 76.88 | 66.5 | 85.98 | 72.86 | 76.97 | 63.92 |
31
+ | **Qwen3-Embedding-0.6B** | 0.6B | 66.33 | 67.45 | 71.40 | 68.74 | 76.42 | 62.58 | 71.03 | 54.52 |
32
+ | **This Model** | 0.6B-W4A16 | 65.21 | 66.30 | 71.36 | 66.12 | 74.96 | 62.63 | 69.10 | 53.65 |
33
 
34
  ## How to use it?
35