I just released Sentence Transformers v4.1; featuring ONNX and OpenVINO backends for rerankers offering 2-3x speedups and improved hard negatives mining which helps prepare stronger training datasets. Details:
text
I just released Sentence Transformers v4.1; featuring ONNX and OpenVINO backends for rerankers offering 2-3x speedups and improved hard negatives mining which helps prepare stronger training datasets. Details:
new_line
new_line
🏎️ ONNX, OpenVINO, Optimization, Quantization
text
🏎️ ONNX, OpenVINO, Optimization, Quantization
new_line
- I've added ONNX and OpenVINO support with just one extra argument: "backend" when loading the CrossEncoder reranker, e.g.: 
text
- I've added ONNX and OpenVINO support with just one extra argument: "backend" when loading the CrossEncoder reranker, e.g.: 
CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="onnx")
`CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="onnx")`
inline_code
new_line
- The 
text
- The 
export_optimized_onnx_model
`export_optimized_onnx_model`
inline_code
, 
text
, 
export_dynamic_quantized_onnx_model
`export_dynamic_quantized_onnx_model`
inline_code
, and 
text
, and 
export_static_quantized_openvino_model
`export_static_quantized_openvino_model`
inline_code
 functions now work with CrossEncoder rerankers, allowing you to optimize (e.g. fusions, gelu approximations, etc.) or quantize (int8 weights) rerankers.
text
 functions now work with CrossEncoder rerankers, allowing you to optimize (e.g. fusions, gelu approximations, etc.) or quantize (int8 weights) rerankers.
new_line
- I've uploaded ~340 ONNX & OpenVINO models for all existing models under the cross-encoder Hugging Face organization. You can use these without having to export when loading.
text
- I've uploaded ~340 ONNX & OpenVINO models for all existing models under the cross-encoder Hugging Face organization. You can use these without having to export when loading.
new_line
new_line
⛏ Improved Hard Negatives Mining
text
⛏ Improved Hard Negatives Mining
new_line
- Added 'absolute_margin' and 'relative_margin' arguments to 
text
- Added 'absolute_margin' and 'relative_margin' arguments to 
mine_hard_negatives
`mine_hard_negatives`
inline_code
. 
text
. 
new_line
- 
text
- 
absolute_margin
`absolute_margin`
inline_code
 ensures that 
text
 ensures that 
sim(query, negative) < sim(query, positive) - absolute_margin
`sim(query, negative) < sim(query, positive) - absolute_margin`
inline_code
, i.e. an absolute margin between the negative & positive similarities.
text
, i.e. an absolute margin between the negative & positive similarities.
new_line
- 
text
- 
relative_margin
`relative_margin`
inline_code
 ensures that 
text
 ensures that 
sim(query, negative) < sim(query, positive) * (1 - relative_margin)
`sim(query, negative) < sim(query, positive) * (1 - relative_margin)`
inline_code
, i.e. a relative margin between the negative & positive similarities.
text
, i.e. a relative margin between the negative & positive similarities.
new_line
- Inspired by the excellent NV-Retriever paper from NVIDIA.
text
- Inspired by the excellent NV-Retriever paper from NVIDIA.
new_line
new_line
And several other small improvements. Check out the full release notes here: 
text
And several other small improvements. Check out the full release notes here: 
https://github.com/UKPLab/sentence-transformers/releases/tag/v4.1.0
https://github.com/UKPLab/sentence-transformers/releases/tag/v4.1.0
link
new_line
new_line
With this release, I introduce near-feature parity between the SentenceTransformer embedding & CrossEncoder reranker models, which I've wanted to do for quite some time! With rerankers very strongly supported now, it's time to look forward to other useful architectures!
text
With this release, I introduce near-feature parity between the SentenceTransformer embedding & CrossEncoder reranker models, which I've wanted to do for quite some time! With rerankers very strongly supported now, it's time to look forward to other useful architectures!
new_line
new_line
I just released Sentence Transformers v4.1; featuring ONNX and OpenVINO backends for rerankers offering 2-3x speedups and improved hard negatives mining which helps prepare stronger training datasets. Details:

🏎️ ONNX, OpenVINO, Optimization, Quantization
- I've added ONNX and OpenVINO support with just one extra argument: "backend" when loading the CrossEncoder reranker, e.g.: `CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="onnx")`
- The `export_optimized_onnx_model`, `export_dynamic_quantized_onnx_model`, and `export_static_quantized_openvino_model` functions now work with CrossEncoder rerankers, allowing you to optimize (e.g. fusions, gelu approximations, etc.) or quantize (int8 weights) rerankers.
- I've uploaded ~340 ONNX & OpenVINO models for all existing models under the cross-encoder Hugging Face organization. You can use these without having to export when loading.

⛏ Improved Hard Negatives Mining
- Added 'absolute_margin' and 'relative_margin' arguments to `mine_hard_negatives`. 
- `absolute_margin` ensures that `sim(query, negative) < sim(query, positive) - absolute_margin`, i.e. an absolute margin between the negative & positive similarities.
- `relative_margin` ensures that `sim(query, negative) < sim(query, positive) * (1 - relative_margin)`, i.e. a relative margin between the negative & positive similarities.
- Inspired by the excellent NV-Retriever paper from NVIDIA.

And several other small improvements. Check out the full release notes here: https://github.com/UKPLab/sentence-transformers/releases/tag/v4.1.0

With this release, I introduce near-feature parity between the SentenceTransformer embedding & CrossEncoder reranker models, which I've wanted to do for quite some time! With rerankers very strongly supported now, it's time to look forward to other useful architectures!


🔥
John6666 yjoonjang tattrongvu dreamhope iberob