--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:9316 - loss:MultipleNegativesRankingLoss base_model: intfloat/multilingual-e5-large widget: - source_sentence: Horn band legwearis a type oflegwear, oftenthighhighs, with ahornedcharacter design along the upper band. sentences: - horn band legwear - head out of frame - sweatpants - source_sentence: When a character is looping the laces of theiruntied shoelacesinto a sturdy bow. sentences: - hair tie - tying footwear - loose necktie - source_sentence: Use this tag if the person's eyewear isremovedfrom their usual place and carried in the hands. If it still rests on the bridge of the nose or head, seeadjusting eyewearand its related tags. sentences: - cow costume - sarong - holding removed eyewear - source_sentence: When both of a character's hands are on another character'sthighs. sentences: - baking - triplets - hands on another's thighs - source_sentence: A long appendage protruding from the lower back. Often covered in fur or scales. A common feature of animal girls. sentences: - tail - grey-framed eyewear - stomach day datasets: - meandyou200175/word_embedding pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@2 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_accuracy@100 - cosine_precision@1 - cosine_precision@2 - cosine_precision@5 - cosine_precision@10 - cosine_precision@100 - cosine_recall@1 - cosine_recall@2 - cosine_recall@5 - cosine_recall@10 - cosine_recall@100 - cosine_ndcg@10 - cosine_mrr@1 - cosine_mrr@2 - cosine_mrr@5 - cosine_mrr@10 - cosine_mrr@100 - cosine_map@100 model-index: - name: SentenceTransformer based on intfloat/multilingual-e5-large results: - task: type: information-retrieval name: Information Retrieval dataset: name: Unknown type: unknown metrics: - type: cosine_accuracy@1 value: 0.8108108108108109 name: Cosine Accuracy@1 - type: cosine_accuracy@2 value: 0.8957528957528957 name: Cosine Accuracy@2 - type: cosine_accuracy@5 value: 0.9382239382239382 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9642857142857143 name: Cosine Accuracy@10 - type: cosine_accuracy@100 value: 0.9932432432432432 name: Cosine Accuracy@100 - type: cosine_precision@1 value: 0.8108108108108109 name: Cosine Precision@1 - type: cosine_precision@2 value: 0.44787644787644787 name: Cosine Precision@2 - type: cosine_precision@5 value: 0.18764478764478765 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09642857142857143 name: Cosine Precision@10 - type: cosine_precision@100 value: 0.009932432432432433 name: Cosine Precision@100 - type: cosine_recall@1 value: 0.8108108108108109 name: Cosine Recall@1 - type: cosine_recall@2 value: 0.8957528957528957 name: Cosine Recall@2 - type: cosine_recall@5 value: 0.9382239382239382 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9642857142857143 name: Cosine Recall@10 - type: cosine_recall@100 value: 0.9932432432432432 name: Cosine Recall@100 - type: cosine_ndcg@10 value: 0.8923095558988695 name: Cosine Ndcg@10 - type: cosine_mrr@1 value: 0.8108108108108109 name: Cosine Mrr@1 - type: cosine_mrr@2 value: 0.8532818532818532 name: Cosine Mrr@2 - type: cosine_mrr@5 value: 0.8649292149292154 name: Cosine Mrr@5 - type: cosine_mrr@10 value: 0.8687695348409635 name: Cosine Mrr@10 - type: cosine_mrr@100 value: 0.8700193430588538 name: Cosine Mrr@100 - type: cosine_map@100 value: 0.8700193430588539 name: Cosine Map@100 --- # SentenceTransformer based on intfloat/multilingual-e5-large This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) on the [word_embedding](https://huggingface.co/datasets/meandyou200175/word_embedding) dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 1024 dimensions - **Similarity Function:** Cosine Similarity - **Training Dataset:** - [word_embedding](https://huggingface.co/datasets/meandyou200175/word_embedding) ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("meandyou200175/e5_large_finetune_word") # Run inference sentences = [ 'A long appendage protruding from the lower back. Often covered in fur or scales. A common feature of animal girls.', 'tail', 'stomach day', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 1024] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:---------------------|:-----------| | cosine_accuracy@1 | 0.8108 | | cosine_accuracy@2 | 0.8958 | | cosine_accuracy@5 | 0.9382 | | cosine_accuracy@10 | 0.9643 | | cosine_accuracy@100 | 0.9932 | | cosine_precision@1 | 0.8108 | | cosine_precision@2 | 0.4479 | | cosine_precision@5 | 0.1876 | | cosine_precision@10 | 0.0964 | | cosine_precision@100 | 0.0099 | | cosine_recall@1 | 0.8108 | | cosine_recall@2 | 0.8958 | | cosine_recall@5 | 0.9382 | | cosine_recall@10 | 0.9643 | | cosine_recall@100 | 0.9932 | | **cosine_ndcg@10** | **0.8923** | | cosine_mrr@1 | 0.8108 | | cosine_mrr@2 | 0.8533 | | cosine_mrr@5 | 0.8649 | | cosine_mrr@10 | 0.8688 | | cosine_mrr@100 | 0.87 | | cosine_map@100 | 0.87 | ## Training Details ### Training Dataset #### word_embedding * Dataset: [word_embedding](https://huggingface.co/datasets/meandyou200175/word_embedding) at [af76b11](https://huggingface.co/datasets/meandyou200175/word_embedding/tree/af76b11c1d93542ca76e864a60b1744d5e02b099) * Size: 9,316 training samples * Columns: query and positive * Approximate statistics based on the first 1000 samples: | | query | positive | |:--------|:-----------------------------------------------------------------------------------|:--------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | query | positive | |:------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------| | Eyewear shaped like a semicircle. | semi-circular eyewear | | A handheld electric appliance used fordryingand styling hair. | hair dryer | | When onebreastis exposed while the other remains covered or confined by clothing. Seebreasts outfor when both breasts are exposed. | one breast out | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Evaluation Dataset #### word_embedding * Dataset: [word_embedding](https://huggingface.co/datasets/meandyou200175/word_embedding) at [af76b11](https://huggingface.co/datasets/meandyou200175/word_embedding/tree/af76b11c1d93542ca76e864a60b1744d5e02b099) * Size: 1,036 evaluation samples * Columns: query and positive * Approximate statistics based on the first 1000 samples: | | query | positive | |:--------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | query | positive | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------| | A machine that manipulates data according to a list of instructions. The ability to store and execute lists of instructions called programs make computers extremely versatile. On Danbooru's images they are most often used fordrawing,playing gamesand accessing theinternet. | computer | | Aplaying cardwith twoclubs. | two of clubs | | Yebisu (ヱビス, Ebisu) is a beer produced bySapporo Breweries. It is one of Japan's oldest brands, first being brewed in Tokyo in 1890 by the Japan Beer Brewery Company. | yebisu | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 5 - `warmup_ratio`: 0.1 - `fp16`: True - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 5 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: True - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `tp_size`: 0 - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | Validation Loss | cosine_ndcg@10 | |:------:|:----:|:-------------:|:---------------:|:--------------:| | -1 | -1 | - | - | 0.7166 | | 0.1715 | 100 | 0.8892 | - | - | | 0.3431 | 200 | 0.1724 | - | - | | 0.5146 | 300 | 0.1783 | - | - | | 0.6861 | 400 | 0.1393 | - | - | | 0.8576 | 500 | 0.1262 | - | - | | 1.0292 | 600 | 0.1046 | - | - | | 1.2007 | 700 | 0.0639 | - | - | | 1.3722 | 800 | 0.0692 | - | - | | 1.5437 | 900 | 0.043 | - | - | | 1.7153 | 1000 | 0.0614 | 0.0819 | 0.8774 | | 1.8868 | 1100 | 0.0538 | - | - | | 2.0583 | 1200 | 0.0414 | - | - | | 2.2298 | 1300 | 0.0146 | - | - | | 2.4014 | 1400 | 0.0164 | - | - | | 2.5729 | 1500 | 0.0225 | - | - | | 2.7444 | 1600 | 0.0215 | - | - | | 2.9160 | 1700 | 0.0271 | - | - | | 3.0875 | 1800 | 0.0202 | - | - | | 3.2590 | 1900 | 0.0194 | - | - | | 3.4305 | 2000 | 0.0144 | 0.0682 | 0.8923 | | 3.6021 | 2100 | 0.0118 | - | - | | 3.7736 | 2200 | 0.0155 | - | - | | 3.9451 | 2300 | 0.0177 | - | - | | 4.1166 | 2400 | 0.0059 | - | - | | 4.2882 | 2500 | 0.0099 | - | - | | 4.4597 | 2600 | 0.0056 | - | - | | 4.6312 | 2700 | 0.0153 | - | - | | 4.8027 | 2800 | 0.0069 | - | - | | 4.9743 | 2900 | 0.01 | - | - | ### Framework Versions - Python: 3.11.11 - Sentence Transformers: 3.4.1 - Transformers: 4.51.1 - PyTorch: 2.5.1+cu124 - Accelerate: 1.3.0 - Datasets: 3.5.0 - Tokenizers: 0.21.0 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```