HancomInSpaceAI commited on
Commit
70d97d4
ยท
verified ยท
1 Parent(s): 421004b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +230 -218
README.md CHANGED
@@ -1,219 +1,231 @@
1
- <div align="center">
2
- <h1><span style="color:#475b9f;">H</span> <span style="color:#ff5733;">i</span> Embed_base_v1
3
- <p>
4
- <p style="font-size: 0.6em;">: Hancom InSpace Embedding Model (Base Type)</p>
5
- </h1>
6
- </div>
7
-
8
-
9
-
10
- <img src="./images/InSpace.png" alt="Alt text" width="100%"/>
11
-
12
-
13
- **Developer: Hancom InSpace**
14
-
15
- **Supported languages: Korean, English**
16
-
17
- **Model Release Date: September 7th, 2025**
18
-
19
- ---
20
-
21
- ## Information
22
-
23
- Introducing `HiEmbed_base`, a lightweight embedding model for vector search from Hancom InSpace.
24
- This model demonstrates outstanding performance in both Korean and English text retrieval, efficiently supporting multilingual environments. When implementing **RAG (Retrieval-Augmented Generation)** in a business setting, it offers high competitiveness in terms of supported languages, model size, speed, and accuracy, allowing for the expectation of excellent results.
25
- Through this model release, we aim to contribute to the activation of the domestic LLM and RAG ecosystem. We hope that many people will freely test it and apply it to various services.
26
- This model was trained with the support of GPU infrastructure from **Gyeongnam Technopark.**
27
-
28
- ## Training
29
-
30
- **Dataset: In-house built dataset (3 million entries)**
31
-
32
- > `HiEmbed_base` is designed to maximize multilingual performance in both Korean and English simultaneously. We constructed an optimized dataset through iterative experiments, and its main features are as follows.
33
- >
34
-
35
- **Data Types**
36
-
37
- - **KO:** QA and documents in the fields of administration, law, news, finance, and science & technology.
38
- - **EN:** Q&A data on various topics, including web searches and community Q&A.
39
-
40
- **Dataset Composition and Scale**
41
-
42
- - **Structure:** Utilizes a triplet structure of query, positive, and negative.
43
-
44
- ```json
45
- {
46
- "query": "question or anchor sentence",
47
- "pos": ["positive sample sentence 1"],
48
- "neg": ["negative sample sentence 1"]
49
- }
50
- ```
51
-
52
- - **Core Processing:** Applied Hard Negative Sampling to enhance the model's discriminative ability.
53
- - **Final Scale:** Training was conducted with a total of 3 million data entries constructed through the above process.
54
- - **Data Split:**
55
- - **Training/Validation Ratio:** 92% / 8%
56
- - **Training Data Language Ratio (KO:EN):** 7:3 (Focused on strengthening Korean performance)
57
- - **Validation Data Language Ratio (KO:EN):** 1:1 (For a balanced performance evaluation of both languages)
58
-
59
- We have put a great deal of effort into the quality of the dataset and the balance of the training data to improve not only Korean but also multilingual support and performance, including English. After testing various ratios, we found that maintaining a 7:3 ratio during training yielded the best results.
60
-
61
- ## Usage
62
-
63
- ### **Install**
64
-
65
- First, install the necessary libraries. For GPU support, ensure you have a compatible CUDA environment.
66
-
67
- ```python
68
- # Common libraries
69
- pip install -U transformers onnx
70
-
71
- # For using Optimum
72
- # For GPU (CUDA)
73
- pip install optimum[onnxruntime-gpu]
74
- # For CPU only
75
- pip install optimum[onnxruntime]
76
- ```
77
-
78
- **Note:** Ensure that the installed version of `onnxruntime-gpu` is compatible with your system's CUDA version. If you encounter errors, you may need to manually install a specific version of `onnxruntime-gpu` that matches your CUDA environment.
79
-
80
- **Running the model on a GPU**
81
-
82
- Set the provider to `CUDAExecutionProvider` and specify the GPU index for the `device`.
83
-
84
- ```python
85
- from optimum.pipelines import pipeline
86
- from transformers import AutoTokenizer
87
- from optimum.onnxruntime import ORTModelForFeatureExtraction
88
-
89
- model_path = "/onnx/.."
90
- sentences = [
91
- "๊ฒฝ์ƒ๋‚จ๋„๋Š” ์‚ฐ๋ถˆ ํ”ผํ•ด๋ฅผ ์ž…์€ 4๊ฐœ ์‹œยท๊ตฐ์— ๋Œ€ํ•ด ๊ธด๊ธ‰ ๋ณต๊ตฌ๋น„ 83์–ต์›์„ ์ง€์›ํ•˜๊ณ , ์ฃผ๋ฏผ ์„ค๋ช…ํšŒ๋ฅผ ํ†ตํ•ด ์ฃผํƒ ๋ณต๊ตฌ ์ผ์ •๊ณผ ์ ˆ์ฐจ๋ฅผ ์•ˆ๋‚ดํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค.",
92
- "ํ•œ์ปด์ธ์ŠคํŽ˜์ด์Šค(๋Œ€ํ‘œ ์ตœ๋ช…์ง„)๋Š” ๊ธฐ์ˆ ์„ฑ ํ‰๊ฐ€๋ฅผ ํ†ต๊ณผํ•˜๊ณ  ํ”„๋ฆฌ IPO๋กœ 125์–ต ์›์„ ์œ ์น˜ํ–ˆ์œผ๋ฉฐ, ์ด๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ์—ฐ๋‚ด ์ฝ”์Šค๋‹ฅ ์ƒ์žฅ ์ ˆ์ฐจ๋ฅผ ๋ณธ๊ฒฉํ™”ํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค."
93
- ]
94
-
95
- # 1. Load model with CUDAExecutionProvider
96
- ort_model = ORTModelForFeatureExtraction.from_pretrained(
97
- model_path,
98
- provider="CUDAExecutionProvider",
99
- local_files_only=True
100
- )
101
- tokenizer = AutoTokenizer.from_pretrained(model_path)
102
-
103
- # 2. Create pipeline on a specific GPU
104
- pipe = pipeline(
105
- task="feature-extraction",
106
- model=ort_model,
107
- tokenizer=tokenizer,
108
- device="cuda:0" # Target the first GPU
109
- )
110
-
111
- # 3. Get embeddings
112
- embeddings = pipe(sentences)
113
- print("Embeddings generated on GPU.")
114
-
115
- ```
116
-
117
- **Running the model on multiple GPUs**
118
-
119
- The optimum pipeline for ONNX Runtime does not automatically parallelize a single request across multiple GPUs. The common approach for multi-GPU usage is to run separate inference processes on different GPUs to handle batches in parallel.
120
-
121
- You can achieve this by creating multiple pipelines, each assigned to a different GPU device.
122
-
123
- ```python
124
- from optimum.pipelines import pipeline
125
- from transformers import AutoTokenizer
126
- from optimum.onnxruntime import ORTModelForFeatureExtraction
127
-
128
- model_path = "/onnx/.."
129
-
130
- # Load the model once
131
- ort_model = ORTModelForFeatureExtraction.from_pretrained(
132
- model_path,
133
- provider="CUDAExecutionProvider",
134
- local_files_only=True
135
- )
136
- tokenizer = AutoTokenizer.from_pretrained(model_path)
137
-
138
- # Create a pipeline for each GPU
139
- pipe_gpu0 = pipeline(task="feature-extraction", model=ort_model, tokenizer=tokenizer, device="cuda:0")
140
- pipe_gpu1 = pipeline(task="feature-extraction", model=ort_model, tokenizer=tokenizer, device="cuda:1")
141
-
142
- # Process different data on each pipeline (e.g., in separate threads/processes)
143
- sentences_batch1 = ["๊ฒฝ์ƒ๋‚จ๋„๋Š” ์‚ฐ๋ถˆ ํ”ผํ•ด๋ฅผ ์ž…์€ 4๊ฐœ ์‹œยท๊ตฐ์— ๋Œ€ํ•ด ๊ธด๊ธ‰ ๋ณต๊ตฌ๋น„ 83์–ต์›์„ ์ง€์›ํ•˜๊ณ , ์ฃผ๋ฏผ ์„ค๋ช…ํšŒ๋ฅผ ํ†ตํ•ด ์ฃผํƒ ๋ณต๊ตฌ ์ผ์ •๊ณผ ์ ˆ์ฐจ๋ฅผ ์•ˆ๋‚ดํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค."]
144
- sentences_batch2 = ["ํ•œ์ปด์ธ์ŠคํŽ˜์ด์Šค(๋Œ€ํ‘œ ์ตœ๋ช…์ง„)๋Š” ๊ธฐ์ˆ ์„ฑ ํ‰๊ฐ€๋ฅผ ํ†ต๊ณผํ•˜๊ณ  ํ”„๋ฆฌ IPO๋กœ 125์–ต ์›์„ ์œ ์น˜ํ–ˆ์œผ๋ฉฐ, ์ด๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ์—ฐ๋‚ด ์ฝ”์Šค๋‹ฅ ์ƒ์žฅ ์ ˆ์ฐจ๋ฅผ ๋ณธ๊ฒฉํ™”ํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค."]
145
-
146
- embeddings1 = pipe_gpu0(sentences_batch1)
147
- embeddings2 = pipe_gpu1(sentences_batch2)
148
-
149
- print("Batch 1 processed on cuda:0.")
150
- print("Batch 2 processed on cuda:1.")
151
- ```
152
-
153
- ## Benchmark
154
-
155
- ### Korean Leaderboard
156
-
157
- | Model | Korean | KLUE-STS | KLUE-TC | Ko-StrategyQA | KorSTS | AutoRAGRetrieval | XPQARetrieval |
158
- | --- | --- | --- | --- | --- | --- | --- | --- |
159
- | bge-m3 | 69.2 | 87.71 | 55.5 | 79.4 | 80.26 | 83.01 | 29.33 |
160
- | BGE-m3-ko | 70.72 | 88.65 | 55.35 | 79.59 | 81.59 | 87.38 | 31.8 |
161
- | HancomInSapce_HiEmbed_base | 71.53 | 87.98 | 61.79 | 79.92 | 81.75 | 85.62 | 32.13 |
162
- | ibm-granite_granite-embedding-107m-multilingual | 57.95 | 73.25 | 48.23 | 70.53 | 70.6 | 68.24 | 16.83 |
163
- | intfloat_multilingual-e5-base | 64.41 | 77.71 | 59.74 | 75.45 | 75.2 | 77.66 | 20.7 |
164
- | intfloat_multilingual-e5-large | 68.14 | 81.58 | 62.09 | 79.82 | 79.24 | 80.66 | 25.45 |
165
- | intfloat_multilingual-e5-large-instruct | 67.11 | 86.98 | 63.61 | 75.68 | 79.64 | 68.08 | 28.64 |
166
- | KURE-v1 | 71.44 | 87.74 | 61.32 | 79.99 | 81.25 | 87.08 | 31.25 |
167
- | sentence-transformers_all-MiniLM-L6-v2 | 15.67 | 22.36 | 19.84 | 1.41 | 42.3 | 6.22 | 1.9 |
168
- | Snowflake_snowflake-arctic-embed-l-v2.0 | 68.69 | 82.55 | 58.98 | 80.46 | 73.81 | 83.86 | 32.47 |
169
- | Snowflake_snowflake-arctic-embed-m | 20.96 | 38.52 | 19.57 | 4.57 | 39.1 | 19.19 | 4.8 |
170
- | Snowflake_snowflake-arctic-embed-s | 17.64 | 26.17 | 20.5 | 4.89 | 33.36 | 16.91 | 4 |
171
-
172
- ### English Leaderboard
173
-
174
- | Model | English | Retrieval | STS | Classification | Clustering | PairClassification | Reranking |
175
- | --- | --- | --- | --- | --- | --- | --- | --- |
176
- | bge-m3 | 28.5 | 54.42 | 80.44 | 63.68 | 42.04 | 84.48 | 55.27 |
177
- | BGE-m3-ko | 28.66 | 55.83 | 81.18 | 63.64 | 43.49 | 83.95 | 55.2 |
178
- | HancomInSapce_HiEmbed_base | 29.07 | 56.22 | 81.37 | 64.45 | 44.83 | 84.8 | 56.21 |
179
- | ibm-granite_granite-embedding-107m-multilingual | 26.3 | 44.77 | 72.55 | 54.26 | 41.82 | 80.29 | 55.59 |
180
- | intfloat_multilingual-e5-base | 27.75 | 48.8 | 75.98 | 61.34 | 44.22 | 83.74 | 54.16 |
181
- | intfloat_multilingual-e5-large | 28.28 | 52.29 | 78.97 | 61.66 | 45.52 | 84.32 | 54.67 |
182
- | intfloat_multilingual-e5-large-instruct | 29.05 | 50.29 | 81 | 62.67 | 49.9 | 82.12 | 55.32 |
183
- | KURE-v1 | 28.77 | 55.8 | 80.74 | 64.16 | 44.12 | 84.56 | 55.71 |
184
- | sentence-transformers_all-MiniLM-L6-v2 | 27.11 | 17.54 | 51.56 | 51.86 | 46.22 | 82.37 | 58.04 |
185
- | Snowflake_snowflake-arctic-embed-l-v2.0 | 28.75 | 57.6 | 75.68 | 60.06 | 47.58 | 83 | 57 |
186
- | Snowflake_snowflake-arctic-embed-m | 26.58 | 22.95 | 51.23 | 49.39 | 47.65 | 75.83 | 57.88 |
187
- | Snowflake_snowflake-arctic-embed-s | 26.87 | 21.5 | 48.07 | 52.19 | 46.79 | 80.11 | 55.93 |
188
-
189
- ### Overall Leaderboard
190
-
191
- | Model | Overall | Korean | English | Retrieval | STS | Classification | Clustering | PairClassification | Reranking |
192
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
193
- | bge-m3 | 63.39 | 69.2 | 28.5 | 54.42 | 80.44 | 63.68 | 42.04 | 84.48 | 55.27 |
194
- | BGE-m3-ko | 63.88 | 70.72 | 28.66 | 55.83 | 81.18 | 63.64 | 43.49 | 83.95 | 55.2 |
195
- | **HancomInSapce_HiEmbed_base** | **64.65** | **71.53** | **29.07** | **56.22** | **81.37** | **64.45** | **44.83** | **84.8** | **56.21** |
196
- | ibm-granite_granite-embedding-107m-multilingual | 58.21 | 57.95 | 26.3 | 44.77 | 72.55 | 54.26 | 41.82 | 80.29 | 55.59 |
197
- | intfloat_multilingual-e5-base | 61.37 | 64.41 | 27.75 | 48.8 | 75.98 | 61.34 | 44.22 | 83.74 | 54.16 |
198
- | intfloat_multilingual-e5-large | 62.91 | 68.14 | 28.28 | 52.29 | 78.97 | 61.66 | 45.52 | 84.32 | 54.67 |
199
- | intfloat_multilingual-e5-large-instruct | 63.55 | 67.11 | 29.05 | 50.29 | 81 | 62.67 | 49.9 | 82.12 | 55.32 |
200
- | KURE-v1 | 64.18 | 71.44 | 28.77 | 55.8 | 80.74 | 64.16 | 44.12 | 84.56 | 55.71 |
201
- | sentence-transformers_all-MiniLM-L6-v2 | 51.27 | 15.67 | 27.11 | 17.54 | 51.56 | 51.86 | 46.22 | 82.37 | 58.04 |
202
- | Snowflake_snowflake-arctic-embed-l-v2.0 | 63.49 | 68.69 | 28.75 | 57.6 | 75.68 | 60.06 | 47.58 | 83 | 57 |
203
- | Snowflake_snowflake-arctic-embed-m | 50.82 | 20.96 | 26.58 | 22.95 | 51.23 | 49.39 | 47.65 | 75.83 | 57.88 |
204
- | Snowflake_snowflake-arctic-embed-s | 50.76 | 17.64 | 26.87 | 21.5 | 48.07 | 52.19 | 46.79 | 80.11 | 55.93 |
205
-
206
- ## About us
207
- <img src="./images/sejong1.webp" alt="Alt text" width="100%"/>
208
-
209
- [Contact us](https://www.inspace.co.kr/)
210
-
211
- ## Caption
212
-
213
- ```json
214
- @misc{
215
- title={HiEmbed_base_v1: Hancom InSpace Embedding Model (Base Type)},
216
- author={JoChanho, KimHajeong},
217
- year={2025}
218
- }
 
 
 
 
 
 
 
 
 
 
 
 
219
  ```
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - ko
5
+ - en
6
+ pipeline_tag: sentence-similarity
7
+ tags:
8
+ - embedding
9
+ - onnx
10
+ - korea
11
+ - korean
12
+ ---
13
+ <div align="center">
14
+ <h1><span style="color:#475b9f;">H</span> <span style="color:#ff5733;">i</span> Embed_base_v1
15
+ <p>
16
+ <p style="font-size: 0.6em;">: Hancom InSpace Embedding Model (Base Type)</p>
17
+ </h1>
18
+ </div>
19
+
20
+
21
+
22
+ <img src="./images/InSpace.png" alt="Alt text" width="100%"/>
23
+
24
+
25
+ **Developer: Hancom InSpace**
26
+
27
+ **Supported languages: Korean, English**
28
+
29
+ **Model Release Date: September 7th, 2025**
30
+
31
+ ---
32
+
33
+ ## Information
34
+
35
+ Introducing `HiEmbed_base`, a lightweight embedding model for vector search from Hancom InSpace.
36
+ This model demonstrates outstanding performance in both Korean and English text retrieval, efficiently supporting multilingual environments. When implementing **RAG (Retrieval-Augmented Generation)** in a business setting, it offers high competitiveness in terms of supported languages, model size, speed, and accuracy, allowing for the expectation of excellent results.
37
+ Through this model release, we aim to contribute to the activation of the domestic LLM and RAG ecosystem. We hope that many people will freely test it and apply it to various services.
38
+ This model was trained with the support of GPU infrastructure from **Gyeongnam Technopark.**
39
+
40
+ ## Training
41
+
42
+ **Dataset: In-house built dataset (3 million entries)**
43
+
44
+ > `HiEmbed_base` is designed to maximize multilingual performance in both Korean and English simultaneously. We constructed an optimized dataset through iterative experiments, and its main features are as follows.
45
+ >
46
+
47
+ **Data Types**
48
+
49
+ - **KO:** QA and documents in the fields of administration, law, news, finance, and science & technology.
50
+ - **EN:** Q&A data on various topics, including web searches and community Q&A.
51
+
52
+ **Dataset Composition and Scale**
53
+
54
+ - **Structure:** Utilizes a triplet structure of query, positive, and negative.
55
+
56
+ ```json
57
+ {
58
+ "query": "question or anchor sentence",
59
+ "pos": ["positive sample sentence 1"],
60
+ "neg": ["negative sample sentence 1"]
61
+ }
62
+ ```
63
+
64
+ - **Core Processing:** Applied Hard Negative Sampling to enhance the model's discriminative ability.
65
+ - **Final Scale:** Training was conducted with a total of 3 million data entries constructed through the above process.
66
+ - **Data Split:**
67
+ - **Training/Validation Ratio:** 92% / 8%
68
+ - **Training Data Language Ratio (KO:EN):** 7:3 (Focused on strengthening Korean performance)
69
+ - **Validation Data Language Ratio (KO:EN):** 1:1 (For a balanced performance evaluation of both languages)
70
+
71
+ We have put a great deal of effort into the quality of the dataset and the balance of the training data to improve not only Korean but also multilingual support and performance, including English. After testing various ratios, we found that maintaining a 7:3 ratio during training yielded the best results.
72
+
73
+ ## Usage
74
+
75
+ ### **Install**
76
+
77
+ First, install the necessary libraries. For GPU support, ensure you have a compatible CUDA environment.
78
+
79
+ ```python
80
+ # Common libraries
81
+ pip install -U transformers onnx
82
+
83
+ # For using Optimum
84
+ # For GPU (CUDA)
85
+ pip install optimum[onnxruntime-gpu]
86
+ # For CPU only
87
+ pip install optimum[onnxruntime]
88
+ ```
89
+
90
+ **Note:** Ensure that the installed version of `onnxruntime-gpu` is compatible with your system's CUDA version. If you encounter errors, you may need to manually install a specific version of `onnxruntime-gpu` that matches your CUDA environment.
91
+
92
+ **Running the model on a GPU**
93
+
94
+ Set the provider to `CUDAExecutionProvider` and specify the GPU index for the `device`.
95
+
96
+ ```python
97
+ from optimum.pipelines import pipeline
98
+ from transformers import AutoTokenizer
99
+ from optimum.onnxruntime import ORTModelForFeatureExtraction
100
+
101
+ model_path = "/onnx/.."
102
+ sentences = [
103
+ "๊ฒฝ์ƒ๋‚จ๋„๋Š” ์‚ฐ๋ถˆ ํ”ผํ•ด๋ฅผ ์ž…์€ 4๊ฐœ ์‹œยท๊ตฐ์— ๋Œ€ํ•ด ๊ธด๊ธ‰ ๋ณต๊ตฌ๋น„ 83์–ต์›์„ ์ง€์›ํ•˜๊ณ , ์ฃผ๋ฏผ ์„ค๋ช…ํšŒ๋ฅผ ํ†ตํ•ด ์ฃผํƒ ๋ณต๊ตฌ ์ผ์ •๊ณผ ์ ˆ์ฐจ๋ฅผ ์•ˆ๋‚ดํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค.",
104
+ "ํ•œ์ปด์ธ์ŠคํŽ˜์ด์Šค(๋Œ€ํ‘œ ์ตœ๋ช…์ง„)๋Š” ๊ธฐ์ˆ ์„ฑ ํ‰๊ฐ€๋ฅผ ํ†ต๊ณผํ•˜๊ณ  ํ”„๋ฆฌ IPO๋กœ 125์–ต ์›์„ ์œ ์น˜ํ–ˆ์œผ๋ฉฐ, ์ด๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ์—ฐ๋‚ด ์ฝ”์Šค๋‹ฅ ์ƒ์žฅ ์ ˆ์ฐจ๋ฅผ ๋ณธ๊ฒฉํ™”ํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค."
105
+ ]
106
+
107
+ # 1. Load model with CUDAExecutionProvider
108
+ ort_model = ORTModelForFeatureExtraction.from_pretrained(
109
+ model_path,
110
+ provider="CUDAExecutionProvider",
111
+ local_files_only=True
112
+ )
113
+ tokenizer = AutoTokenizer.from_pretrained(model_path)
114
+
115
+ # 2. Create pipeline on a specific GPU
116
+ pipe = pipeline(
117
+ task="feature-extraction",
118
+ model=ort_model,
119
+ tokenizer=tokenizer,
120
+ device="cuda:0" # Target the first GPU
121
+ )
122
+
123
+ # 3. Get embeddings
124
+ embeddings = pipe(sentences)
125
+ print("Embeddings generated on GPU.")
126
+
127
+ ```
128
+
129
+ **Running the model on multiple GPUs**
130
+
131
+ The optimum pipeline for ONNX Runtime does not automatically parallelize a single request across multiple GPUs. The common approach for multi-GPU usage is to run separate inference processes on different GPUs to handle batches in parallel.
132
+
133
+ You can achieve this by creating multiple pipelines, each assigned to a different GPU device.
134
+
135
+ ```python
136
+ from optimum.pipelines import pipeline
137
+ from transformers import AutoTokenizer
138
+ from optimum.onnxruntime import ORTModelForFeatureExtraction
139
+
140
+ model_path = "/onnx/.."
141
+
142
+ # Load the model once
143
+ ort_model = ORTModelForFeatureExtraction.from_pretrained(
144
+ model_path,
145
+ provider="CUDAExecutionProvider",
146
+ local_files_only=True
147
+ )
148
+ tokenizer = AutoTokenizer.from_pretrained(model_path)
149
+
150
+ # Create a pipeline for each GPU
151
+ pipe_gpu0 = pipeline(task="feature-extraction", model=ort_model, tokenizer=tokenizer, device="cuda:0")
152
+ pipe_gpu1 = pipeline(task="feature-extraction", model=ort_model, tokenizer=tokenizer, device="cuda:1")
153
+
154
+ # Process different data on each pipeline (e.g., in separate threads/processes)
155
+ sentences_batch1 = ["๊ฒฝ์ƒ๋‚จ๋„๋Š” ์‚ฐ๋ถˆ ํ”ผํ•ด๋ฅผ ์ž…์€ 4๊ฐœ ์‹œยท๊ตฐ์— ๋Œ€ํ•ด ๊ธด๊ธ‰ ๋ณต๊ตฌ๋น„ 83์–ต์›์„ ์ง€์›ํ•˜๊ณ , ์ฃผ๋ฏผ ์„ค๋ช…ํšŒ๋ฅผ ํ†ตํ•ด ์ฃผํƒ ๋ณต๊ตฌ ์ผ์ •๊ณผ ์ ˆ์ฐจ๋ฅผ ์•ˆ๋‚ดํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค."]
156
+ sentences_batch2 = ["ํ•œ์ปด์ธ์ŠคํŽ˜์ด์Šค(๋Œ€ํ‘œ ์ตœ๋ช…์ง„)๋Š” ๊ธฐ์ˆ ์„ฑ ํ‰๊ฐ€๋ฅผ ํ†ต๊ณผํ•˜๊ณ  ํ”„๋ฆฌ IPO๋กœ 125์–ต ์›์„ ์œ ์น˜ํ–ˆ์œผ๋ฉฐ, ์ด๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ์—ฐ๋‚ด ์ฝ”์Šค๋‹ฅ ์ƒ์žฅ ์ ˆ์ฐจ๋ฅผ ๋ณธ๊ฒฉํ™”ํ•  ๊ณ„ํš์ด๋ผ๊ณ  ๋ฐํ˜”๋‹ค."]
157
+
158
+ embeddings1 = pipe_gpu0(sentences_batch1)
159
+ embeddings2 = pipe_gpu1(sentences_batch2)
160
+
161
+ print("Batch 1 processed on cuda:0.")
162
+ print("Batch 2 processed on cuda:1.")
163
+ ```
164
+
165
+ ## Benchmark
166
+
167
+ ### Korean Leaderboard
168
+
169
+ | Model | Korean | KLUE-STS | KLUE-TC | Ko-StrategyQA | KorSTS | AutoRAGRetrieval | XPQARetrieval |
170
+ | --- | --- | --- | --- | --- | --- | --- | --- |
171
+ | bge-m3 | 69.2 | 87.71 | 55.5 | 79.4 | 80.26 | 83.01 | 29.33 |
172
+ | BGE-m3-ko | 70.72 | 88.65 | 55.35 | 79.59 | 81.59 | 87.38 | 31.8 |
173
+ | HancomInSapce_HiEmbed_base | 71.53 | 87.98 | 61.79 | 79.92 | 81.75 | 85.62 | 32.13 |
174
+ | ibm-granite_granite-embedding-107m-multilingual | 57.95 | 73.25 | 48.23 | 70.53 | 70.6 | 68.24 | 16.83 |
175
+ | intfloat_multilingual-e5-base | 64.41 | 77.71 | 59.74 | 75.45 | 75.2 | 77.66 | 20.7 |
176
+ | intfloat_multilingual-e5-large | 68.14 | 81.58 | 62.09 | 79.82 | 79.24 | 80.66 | 25.45 |
177
+ | intfloat_multilingual-e5-large-instruct | 67.11 | 86.98 | 63.61 | 75.68 | 79.64 | 68.08 | 28.64 |
178
+ | KURE-v1 | 71.44 | 87.74 | 61.32 | 79.99 | 81.25 | 87.08 | 31.25 |
179
+ | sentence-transformers_all-MiniLM-L6-v2 | 15.67 | 22.36 | 19.84 | 1.41 | 42.3 | 6.22 | 1.9 |
180
+ | Snowflake_snowflake-arctic-embed-l-v2.0 | 68.69 | 82.55 | 58.98 | 80.46 | 73.81 | 83.86 | 32.47 |
181
+ | Snowflake_snowflake-arctic-embed-m | 20.96 | 38.52 | 19.57 | 4.57 | 39.1 | 19.19 | 4.8 |
182
+ | Snowflake_snowflake-arctic-embed-s | 17.64 | 26.17 | 20.5 | 4.89 | 33.36 | 16.91 | 4 |
183
+
184
+ ### English Leaderboard
185
+
186
+ | Model | English | Retrieval | STS | Classification | Clustering | PairClassification | Reranking |
187
+ | --- | --- | --- | --- | --- | --- | --- | --- |
188
+ | bge-m3 | 28.5 | 54.42 | 80.44 | 63.68 | 42.04 | 84.48 | 55.27 |
189
+ | BGE-m3-ko | 28.66 | 55.83 | 81.18 | 63.64 | 43.49 | 83.95 | 55.2 |
190
+ | HancomInSapce_HiEmbed_base | 29.07 | 56.22 | 81.37 | 64.45 | 44.83 | 84.8 | 56.21 |
191
+ | ibm-granite_granite-embedding-107m-multilingual | 26.3 | 44.77 | 72.55 | 54.26 | 41.82 | 80.29 | 55.59 |
192
+ | intfloat_multilingual-e5-base | 27.75 | 48.8 | 75.98 | 61.34 | 44.22 | 83.74 | 54.16 |
193
+ | intfloat_multilingual-e5-large | 28.28 | 52.29 | 78.97 | 61.66 | 45.52 | 84.32 | 54.67 |
194
+ | intfloat_multilingual-e5-large-instruct | 29.05 | 50.29 | 81 | 62.67 | 49.9 | 82.12 | 55.32 |
195
+ | KURE-v1 | 28.77 | 55.8 | 80.74 | 64.16 | 44.12 | 84.56 | 55.71 |
196
+ | sentence-transformers_all-MiniLM-L6-v2 | 27.11 | 17.54 | 51.56 | 51.86 | 46.22 | 82.37 | 58.04 |
197
+ | Snowflake_snowflake-arctic-embed-l-v2.0 | 28.75 | 57.6 | 75.68 | 60.06 | 47.58 | 83 | 57 |
198
+ | Snowflake_snowflake-arctic-embed-m | 26.58 | 22.95 | 51.23 | 49.39 | 47.65 | 75.83 | 57.88 |
199
+ | Snowflake_snowflake-arctic-embed-s | 26.87 | 21.5 | 48.07 | 52.19 | 46.79 | 80.11 | 55.93 |
200
+
201
+ ### Overall Leaderboard
202
+
203
+ | Model | Overall | Korean | English | Retrieval | STS | Classification | Clustering | PairClassification | Reranking |
204
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
205
+ | bge-m3 | 63.39 | 69.2 | 28.5 | 54.42 | 80.44 | 63.68 | 42.04 | 84.48 | 55.27 |
206
+ | BGE-m3-ko | 63.88 | 70.72 | 28.66 | 55.83 | 81.18 | 63.64 | 43.49 | 83.95 | 55.2 |
207
+ | **HancomInSapce_HiEmbed_base** | **64.65** | **71.53** | **29.07** | **56.22** | **81.37** | **64.45** | **44.83** | **84.8** | **56.21** |
208
+ | ibm-granite_granite-embedding-107m-multilingual | 58.21 | 57.95 | 26.3 | 44.77 | 72.55 | 54.26 | 41.82 | 80.29 | 55.59 |
209
+ | intfloat_multilingual-e5-base | 61.37 | 64.41 | 27.75 | 48.8 | 75.98 | 61.34 | 44.22 | 83.74 | 54.16 |
210
+ | intfloat_multilingual-e5-large | 62.91 | 68.14 | 28.28 | 52.29 | 78.97 | 61.66 | 45.52 | 84.32 | 54.67 |
211
+ | intfloat_multilingual-e5-large-instruct | 63.55 | 67.11 | 29.05 | 50.29 | 81 | 62.67 | 49.9 | 82.12 | 55.32 |
212
+ | KURE-v1 | 64.18 | 71.44 | 28.77 | 55.8 | 80.74 | 64.16 | 44.12 | 84.56 | 55.71 |
213
+ | sentence-transformers_all-MiniLM-L6-v2 | 51.27 | 15.67 | 27.11 | 17.54 | 51.56 | 51.86 | 46.22 | 82.37 | 58.04 |
214
+ | Snowflake_snowflake-arctic-embed-l-v2.0 | 63.49 | 68.69 | 28.75 | 57.6 | 75.68 | 60.06 | 47.58 | 83 | 57 |
215
+ | Snowflake_snowflake-arctic-embed-m | 50.82 | 20.96 | 26.58 | 22.95 | 51.23 | 49.39 | 47.65 | 75.83 | 57.88 |
216
+ | Snowflake_snowflake-arctic-embed-s | 50.76 | 17.64 | 26.87 | 21.5 | 48.07 | 52.19 | 46.79 | 80.11 | 55.93 |
217
+
218
+ ## About us
219
+ <img src="./images/sejong1.webp" alt="Alt text" width="100%"/>
220
+
221
+ [Contact us](https://www.inspace.co.kr/)
222
+
223
+ ## Caption
224
+
225
+ ```json
226
+ @misc{
227
+ title={HiEmbed_base_v1: Hancom InSpace Embedding Model (Base Type)},
228
+ author={JoChanho, KimHajeong},
229
+ year={2025}
230
+ }
231
  ```