SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2 on the csv dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-mpnet-base-v2
Maximum Sequence Length: 384 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- csv

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Gammellund Ejendomme is a Danish real estate company based in Odense, Denmark. The company specializes in property development, management, and sales services. As of 2024, it employs one person and has reported total assets of approximately 2.9 million DKK. The company is led by Director Brian Gammellund Rasmussen and was founded on October 9, 2020.',
    "KERT INVEST ApS, established on February 6, 2014, is a Danish private limited company based in Helsingør. The company specializes in purchasing, renovating, and selling real estate, as well as trading securities and related activities. ([find-virksomhed.dk](https://find-virksomhed.dk/firma/kert-invest-aps-35658173?utm_source=openai)) The company's registered address is Grønnehavevej 7, 1, 3000 Helsingør. ([lei.bloomberg.com](https://lei.bloomberg.com/gleifs/view/549300IRNKZTEYCVK378?utm_source=openai))",
    "GANNI is a Danish contemporary fashion brand founded in 2000 by Frans Truelsen and revitalized in 2009 by husband-and-wife duo Ditte and Nicolaj Reffstrup. ([en.wikipedia.org](https://en.wikipedia.org/wiki/Ganni?utm_source=openai)) The brand offers a wide range of women's apparel, footwear, eyewear, bags, jewelry, and accessories, embodying a playful and effortless aesthetic that redefines Scandinavian style. ([fashionunited.com](https://fashionunited.com/companies/ganni?utm_source=openai)) GANNI is committed to responsible practices, striving to make environmentally friendly choices and improve daily. ([kristak.com](https://kristak.com/pages/ganni?utm_source=openai))",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.9344

Training Details

Training Dataset

csv

Dataset: csv
Size: 27,191 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 3 tokens mean: 105.8 tokens max: 335 tokens	min: 3 tokens mean: 105.7 tokens max: 361 tokens	min: 3 tokens mean: 108.92 tokens max: 383 tokens

Samples:

anchor	positive	negative
ES Holding Aalborg II was a non-financial holding company based in Støvring, Denmark. Established on November 20, 2019, the company was dissolved after a demerger on December 20, 2019. Its primary purpose was to hold shares in subsidiaries and associated companies, engage in investment and financing activities, and conduct other related business as deemed appropriate by management. The company was registered with a capital of 200,000 DKK and was managed by director Ejner Sørensen. The registered address was Guldbækvej 116, 9530 Støvring, Denmark.	`Malver Holding is a Danish non-financial holding company established on October 9, 2018. Based in Copenhagen, it focuses on investment and holding activities. The company is solely owned and directed by Nicklas Malver, who holds 100% ownership and voting rights. As of 2023, Malver Holding reported a net profit of DKK 437,389 and total assets amounting to DKK 3,889,000. The company's registered address is Hjørringgade 1, 3. tv., 2100 København Ø.`	`Research Infrastructure Consultancy Services is a Danish firm specializing in providing expert guidance and support for the development and management of research infrastructures. Their services encompass strategic planning, project management, and operational optimization to enhance the efficiency and effectiveness of research facilities. By collaborating closely with clients, they aim to tailor solutions that meet the unique needs of various research institutions.`
SIGNCONCEPT is a Danish company specializing in the signage and advertising industry. Established in 2006, the company operates from its headquarters at Industrivej 60, 6740 Bramming, Denmark. SIGNCONCEPT offers a range of products and services related to signs and advertising materials, catering to various business needs. The company is registered under CVR number 30502590 and has a workforce of approximately 4 employees. For more information, visit their official website at http://www.signconcept.dk.	`Fleet Complete Danmark specializes in fleet management solutions, offering GPS tracking, electronic logbooks, and task management systems to optimize vehicle fleets and mobile workforces. Their services aim to enhance performance, reduce fuel consumption, and integrate seamlessly with existing operational systems.`	Lidemark Kirke is a historic church located in Bjæverskov, Denmark. Built in the 12th century in Romanesque style, the original structure comprises an apse, chancel, and nave. Around 1500, additions such as a porch, sacristy, and tower were incorporated. The church is primarily constructed from chalk and split fieldstone. Notable features include an altarpiece with two large columns and a painting titled "Christ in the Resurrection" by F. Storck from 1860. The tower houses a beautifully crafted organ built by K. Olsen in 1870, and the church has two bells dating from 1749 and 1842. A Renaissance gravestone commemorates Hartvig Høcken, a local nobleman who passed away in 1595. The church is part of a collaborative network with Bjæverskov, Gørslev, and Vollerslev churches, sharing clergy and a parish hall.
`Indian Guro ApS was a Danish company established in 2017. The company was dissolved after bankruptcy in 2023. (paqle.dk)`	`BEG BESLAGSMEDIE ApS was a Danish company established in 2014, specializing in services related to livestock breeding. The company was dissolved after bankruptcy in August 2023. (paqle.dk)`	`Gilleleje Lægecenter is a medical clinic located in Gilleleje, Denmark, offering same-day consultations for various health concerns. Patients can schedule appointments electronically via the clinic's website or the 'Minlæge' app, or by phone. The clinic provides both in-person and video consultations, emphasizing prompt and accessible healthcare services. (xn--gillelejelgecenter-xub.dk)`

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.COSINE",
    "triplet_margin": 0.4
}

Evaluation Dataset

csv

Dataset: csv
Size: 1,432 evaluation samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 3 tokens mean: 106.0 tokens max: 325 tokens	min: 3 tokens mean: 106.5 tokens max: 384 tokens	min: 3 tokens mean: 107.84 tokens max: 384 tokens

Samples:

anchor	positive	negative
Casela ApS is a Danish holding company established on June 17, 2004, located at Haugesundvej 1, 2850 Nærum. The company primarily functions as a non-financial holding entity, owning capital interests in other companies. As of 2023, Casela ApS reported a net profit of 389,125 DKK and an equity of 10,192,000 DKK. The company is led by Director Klaus Kastrup-Larsen, who has been in position since November 7, 2023. The sole owner is Lasse Bo Steenholt, holding 100% of the shares and voting rights since June 17, 2004.	EILKAER HOLDING is a Danish private limited company (Anpartsselskab) established on May 27, 2014. The company is located at Rejnstrupvej 15, 4250 Fuglebjerg, Denmark. Its primary purpose is to own shares and equity interests in other capital companies, manage assets, and engage in related activities as deemed appropriate by the management. The company is led by Director Thomas Bojesen Eilkær, who holds 100% ownership and voting rights. As of 2023, EILKAER HOLDING reported a gross profit of DKK -16,237 and a net income of DKK -4,768. The company is active and operates in the non-financial holding companies industry.	Vesterled Frugtplantage, located on Fejø Island in Denmark, specializes in cultivating high-quality apples, pears, and plums. Benefiting from Fejø's favorable climate, the plantation produces fruit known for its exceptional taste and quality. To ensure freshness, Vesterled Frugtplantage operates its own storage and packing facilities, delivering freshly picked fruit from early August. The plantation adheres to both organic farming practices and the principles of Dansk I.P., minimizing chemical use for the benefit of consumers and the environment.
S/I Margrethe Hjemmet is a private nursing home located in the heart of Roskilde, Denmark. The facility focuses on promoting active aging for both body and soul, providing a harmonious environment for its residents. With 44 apartments spread over two floors, each unit includes a private bathroom and wardrobe, and most feature a terrace or balcony. The home offers various amenities such as a cultural center, dining room, garden, workshop, hair salon, wellness room, and exercise equipment. Emphasizing the importance of family involvement, Margrethe Hjemmet views relatives as valuable resources and staff as catalysts for a meaningful, social, and active elderly life.	`Medarbejderfond for ansatte i ISS Facility Services is a foundation established on December 31, 2005, located at Gyngemose Parkvej 50, 2860 Søborg, Denmark. The foundation operates within the industry of general building cleaning services. As of now, there is no official website registered for this organization.`	Manbook.dk is a Danish company specializing in providing flexible staffing solutions across Denmark. They offer temporary workers for various tasks, including accounting, legal assignments, transportation, and security services. Their services are available 24/7, with the ability to dispatch personnel within two hours. Manbook.dk emphasizes creating a secure environment for both clients and employees, handling administrative tasks such as payroll, pensions, and holidays. Their office is located at Vallensbækvej 6, 2605 Brøndby, Denmark.
Børnehuset Goethesgade is a self-governing, age-integrated daycare institution located in Sønderborg, Denmark. (boernehuset-goethesgade.aula.dk) Established in 1993, it offers a nurturing environment for children aged 0-6 years, comprising a nursery ('bobler') with 18 places and a kindergarten ('stjerner') with 38 places. (boernehuset-goethesgade.aula.dk) The institution emphasizes small group activities to cater to individual child development and foster strong peer relationships. (boernehuset-goethesgade.aula.dk) Situated centrally, it leverages its proximity to nature and the local community to enhance children's daily experiences. (boernehuset-goethesgade.aula.dk)	Horsens Gymnasium & HF is an educational institution located in Horsens, Denmark, offering both the general upper secondary education (STX) and the higher preparatory examination (HF). The school provides a range of study programs, including music, biology and chemistry, social sciences, mathematics, physics, chemistry, geoscience, language studies, and biotechnology. It emphasizes a broad educational foundation, preparing students for further education. The institution also boasts an impressive art collection featuring works by artists such as Kasper Bonnén, Michael Kvium, Cathrine Raben Davidsen, and Poul Anker Bech. (horsens-gym.dk)	`Of Holding is a Danish company based in Aalborg SØ, Nordjylland, specializing in the management of companies and enterprises, particularly as a holding company. The key principal is Ole Frøkjær. Further details about the company's operations and services are not publicly available.`

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.COSINE",
    "triplet_margin": 0.4
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 6
per_device_eval_batch_size: 6
gradient_accumulation_steps: 3
num_train_epochs: 2
warmup_ratio: 0.1
fp16: True
dataloader_pin_memory: False

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 6
per_device_eval_batch_size: 6
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 3
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: False
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	cosine_accuracy
-1	-1	-	-	0.8275
0.0662	100	0.4326	-	-
0.1324	200	0.2973	-	-
0.1655	250	-	0.0902	0.9141
0.1986	300	0.2914	-	-
0.2648	400	0.305	-	-
0.3310	500	0.2878	0.0920	0.9092
0.3972	600	0.308	-	-
0.4634	700	0.2722	-	-
0.4965	750	-	0.0805	0.9218
0.5296	800	0.2591	-	-
0.5958	900	0.2564	-	-
0.6620	1000	0.245	0.0815	0.9197
0.7282	1100	0.2395	-	-
0.7944	1200	0.2559	-	-
0.8274	1250	-	0.0818	0.9232
0.8605	1300	0.2581	-	-
0.9267	1400	0.2692	-	-
0.9929	1500	0.2544	0.0738	0.9302
1.0589	1600	0.2001	-	-
1.1251	1700	0.2112	-	-
1.1582	1750	-	0.0729	0.9302
1.1913	1800	0.1926	-	-
1.2575	1900	0.1801	-	-
1.3237	2000	0.1684	0.0706	0.9267
1.3899	2100	0.1831	-	-
1.4561	2200	0.1963	-	-
1.4892	2250	-	0.0719	0.9281
1.5223	2300	0.1878	-	-
1.5885	2400	0.2028	-	-
1.6547	2500	0.2045	0.0685	0.9323
1.7209	2600	0.1853	-	-
1.7871	2700	0.1793	-	-
1.8202	2750	-	0.0665	0.9344
1.8533	2800	0.1772	-	-
1.9195	2900	0.1722	-	-
1.9857	3000	0.1797	0.0658	0.9344

Framework Versions

Python: 3.13.2
Sentence Transformers: 4.1.0
Transformers: 4.52.1
PyTorch: 2.7.0+cu126
Accelerate: 1.7.0
Datasets: 3.6.0
Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

FORNstudio
/

brine