jo-mengr/mmcontext-cg_3_5k-cg_3_5k-nla-biomodern-None-mixed-geneformer-text_50-v2

SentenceTransformer based on NeuML/bioclinical-modernbert-base-embeddings

This is a sentence-transformers model finetuned from NeuML/bioclinical-modernbert-base-embeddings on the cellxgene_pseudo_bulk_3_5k_multiplets_natural_language_annotation and gene_description datasets. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: NeuML/bioclinical-modernbert-base-embeddings
• Maximum Sequence Length: None tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Datasets:
  • cellxgene_pseudo_bulk_3_5k_multiplets_natural_language_annotation
  • gene_description
• Language: code

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): MMContextEncoder(
    (text_encoder): ModernBertModel(
      (embeddings): ModernBertEmbeddings(
        (tok_embeddings): Embedding(50368, 768, padding_idx=50283)
        (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
        (drop): Dropout(p=0.0, inplace=False)
      )
      (layers): ModuleList(
        (0): ModernBertEncoderLayer(
          (attn_norm): Identity()
          (attn): ModernBertAttention(
            (Wqkv): Linear(in_features=768, out_features=2304, bias=False)
            (rotary_emb): ModernBertRotaryEmbedding()
            (Wo): Linear(in_features=768, out_features=768, bias=False)
            (out_drop): Identity()
          )
          (mlp_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (mlp): ModernBertMLP(
            (Wi): Linear(in_features=768, out_features=2304, bias=False)
            (act): GELUActivation()
            (drop): Dropout(p=0.0, inplace=False)
            (Wo): Linear(in_features=1152, out_features=768, bias=False)
          )
        )
        (1-21): 21 x ModernBertEncoderLayer(
          (attn_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (attn): ModernBertAttention(
            (Wqkv): Linear(in_features=768, out_features=2304, bias=False)
            (rotary_emb): ModernBertRotaryEmbedding()
            (Wo): Linear(in_features=768, out_features=768, bias=False)
            (out_drop): Identity()
          )
          (mlp_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (mlp): ModernBertMLP(
            (Wi): Linear(in_features=768, out_features=2304, bias=False)
            (act): GELUActivation()
            (drop): Dropout(p=0.0, inplace=False)
            (Wo): Linear(in_features=1152, out_features=768, bias=False)
          )
        )
      )
      (final_norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
    )
    (pooling): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
    (omics_encoder): MiniOmicsModel(
      (embeddings): Embedding(745, 512, padding_idx=0)
    )
  )
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jo-mengr/mmcontext-cg_3_5k-cg_3_5k-nla-biomodern-None-mixed-geneformer-text_50-v2")
# Run inference
sentences = [
    'ACTB TMSB4X FTH1 TPT1 TMSB10 MS4A1 CD52 FTL PTMA RPS29 CD79A HLA-DRB5 VIM VPREB3 RPL36A CD82 PPP3CC BTG1 ZBTB20 H4C3 MIR142HG MED24 REST LAT2 STK4 DNAJC3 PDCD5 MMD TFEB PLEK RPS4Y1 IFITM3 C1orf162 PIP4K2A USP25 CBR1 ARPC5 PSIP1 TMC8 ZNF581 QARS1 MAD1L1 CD22 RTF2 UBA6 ADSS2 HERPUD1 SYNE2 EIF2AK2 ST6GAL1',
    "This measurement was conducted with 10x 5' v1. Memory B cell derived from a 3-year-old male with recurrent tonsillitis, expressing IGHJ6*04, IGHV3-21*01, IGKV3-20/IGKJ2, IGKC, and IgG1.",
    "This measurement was conducted with 10x 5' v1. Plasmablast cells derived from a 6-year-old male with recurrent tonsillitis, enriched for memory B cells.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9998, 0.9999],
#         [0.9998, 1.0000, 1.0000],
#         [0.9999, 1.0000, 1.0000]])

Training Details

Training Datasets

cellxgene_pseudo_bulk_3_5k_multiplets_natural_language_annotation

• Dataset: cellxgene_pseudo_bulk_3_5k_multiplets_natural_language_annotation at fe9561c
• Size: 2,331 training samples
• Columns: anchor, positive, negative_1, and negative_2
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative_1	negative_2
  type	string	string	string	string
  details	min: 137 tokens mean: 158.25 tokens max: 192 tokens	min: 22 tokens mean: 52.77 tokens max: 171 tokens	min: 21 tokens mean: 52.25 tokens max: 171 tokens	min: 137 tokens mean: 158.03 tokens max: 190 tokens

• Samples:

  anchor	positive	negative_1	negative_2
  TAGLN MT2A ADIRF ACTA2 MYL9 MT-ND1 IGFBP7 FOS VIM S100A6 MT1E JUN HES4 CALD1 TPM1 FTL SPARCL1 IFITM3 RERGL IGFBP5 DUSP1 RGS5 FRZB PPP1R14A S100A4 CD9 TIMP3 CST3 CCN2 JCHAIN GLUL CXCL14 GSN NEAT1 FKBP5 RGCC CRYAB ZBTB16 SPARC CLU APOE NR4A2 A2M NDUFA4L2 TFPI LAMP2 TAC1 MGST1 IL32 ANLN	This measurement was conducted with 10x 3' v2. Colon pericytes from a male individual in his eighth decade, characterized as Pericytes RERGL NTRK2.	This measurement was conducted with 10x 3' v3. Sample is a lymphocyte cell type, specifically lymphatics, located in the lamina propria of mucosa of colon, taken from a female in her third decade of life with Crohn's disease.	TIMP1 F3 CXCL14 MT-ND1 IGFBP7 MT2A VIM IFITM3 FTL CST3 S100A6 CALD1 COL1A2 A2M C3 JUN MT-ND4L ENG CD74 NEAT1 IL32 MYL9 SAT1 COL4A2 STOM RGS10 IFI16 TPM1 TAGLN GJA1 DENND2B MT1E FOS LHFPL6 ITGA1 TFPI SEMA3A FNDC3B CPQ SPARC G0S2 GNG11 GSN RUNX1 TGFBR2 IFNGR1 ADAM28 PMP22 IGFBP5 WLS
  PLP1 ST18 MBP MAN2A1 RNF220 MOBP LINC00609 PDE1A PIP4K2A CDH20 TMEM144 NCKAP5 TF GAB1 FRMD4B PLEKHH1 PTGDS C10orf90 CDK18 ENPP2 DOCK5 LMCD1-AS1 LINC01505 BCAS1 FAM107B SYNJ2 ATP10B PDE1C UGT8 FBXL7 GRM3 NEAT1 ANLN VRK2 ARAP2 CLDND1 SCD POLR2F CHN2 TLE4 PCSK6 ABCA8 SHROOM4 CERCAM RAD51B KCNH8 LPAR1 DBNDD2 LINC01170 LINC00639	This measurement was conducted with 10x 3' v3. Oligodendrocyte cells from the hippocampal formation of a 29-year-old male, specifically from the Head of hippocampus (HiH) - Uncal DG-CA4 region, with a region of interest at Human DGU-CA4Upy.	This measurement was conducted with 10x 3' v3. Fibroblast cells from the hippocampal formation of a 42-year-old male European donor, specifically from the Head of hippocampus (HiH) - Uncal DG-CA4 region.	RELN ADARB2 CXCL14 NXPH1 DLX6-AS1 GRIK1 PCDH15 GAD2 DPP10 ZMAT4 PLD5 NRG1 HS3ST5 SV2C GRM8 TOX3 UTRN FBXL7 EGFR POU6F2 CLSTN2 IL1RAPL2 ZBTB16 RGS6 GRIN3A COL5A2 COL21A1 SST AP1S2 ADAMTS9-AS2 SEMA6A STMN1 NXPH2 KIT RGS12 SLC24A3 LAMA3 SERPINE2 TRPC3 JAM2 CHST11 PRELID2 ARL4C COL25A1 IGF1 ANKRD44 PCSK5 PPFIBP1 EYA4 WLS
  DPP10 LINC01088 NFIA SOX6 PARD3B RMST CASC15 AGBL1 HPSE2 MAML2 KANK1 GLIS3 ADGRV1 WLS ITGA2 LMCD1-AS1 LINC02055 TCF7L2 GRAMD2B ROR1 SLC1A3 EYA4 CLU FOXP2 GREB1L ANKFN1 MYO1E COLEC12 ZFHX4 PLCH1 CD36 PLCE1 ARHGAP18 CHD7 RAD51B RNF220 NEAT1 PREX2 PCSK5 CDH20 SLC1A2 ACSS3 RFX4 ITPR2 ENOSF1 IQGAP2 CREB5 GJA1 UTRN RGS12	This measurement was conducted with 10x 3' v3. Ependymal cell from the thalamic complex (thalamus (THM) - medial nuclear complex of thalamus (MNC) - mediodorsal nucleus of thalamus + reuniens nucleus (medioventral nucleus) of thalamus - MD + Re) of a 50-year-old male with European ethnicity.	This measurement was conducted with 10x 3' v3. Neuron from the thalamic complex (thalamus, medial nuclear complex, mediodorsal nucleus, and reuniens nucleus) of a 50-year-old male human, identified as midbrain-derived inhibitory.	DPP10 NRG1 NXPH1 COL25A1 CPNE4 GPC5 FOXP2 RNF220 ADAMTSL1 TCF7L2 LHFPL3 MT-ND1 ZFHX3 ITPR2 RMST STXBP6 KCNH5 TOX GRIK1 TTN PLXDC2 HPSE2 SPHKAP PPFIBP1 CDH6 TMEM163 ZMAT4 LINC03000 HS3ST5 VAV3 TOX3 ANKFN1 MARCHF3 WLS CA10 VWC2L MAML2 STMN1 PCDH15 ADARB2 RERGL PROX1 COL21A1 CCBE1 SV2B GRM3 ANKRD44 CLMN SLC24A3 GRIN3A

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

gene_description

• Dataset: gene_description at 338d1c3
• Size: 199 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 199 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 5.53 tokens max: 10 tokens	min: 8 tokens mean: 90.71 tokens max: 268 tokens

• Samples:

  anchor	positive
  A1BG	The protein encoded by this gene is a plasma glycoprotein of unknown function. The protein shows sequence similarity to the variable regions of some immunoglobulin supergene family member proteins. [provided by RefSeq, Jul 2008]
  A1BG-AS1	A1BG antisense RNA 1
  A1CF	Mammalian apolipoprotein B mRNA undergoes site-specific C to U deamination, which is mediated by a multi-component enzyme complex containing a minimal core composed of APOBEC-1 and a complementation factor encoded by this gene. The gene product has three non-identical RNA recognition motifs and belongs to the hnRNP R family of RNA-binding proteins. It has been proposed that this complementation factor functions as an RNA-binding subunit and docks APOBEC-1 to deaminate the upstream cytidine. Studies suggest that the protein may also be involved in other RNA editing or RNA processing events. Several transcript variants encoding a few different isoforms have been found for this gene. [provided by RefSeq, Nov 2010]

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Datasets

cellxgene_pseudo_bulk_3_5k_multiplets_natural_language_annotation

• Dataset: cellxgene_pseudo_bulk_3_5k_multiplets_natural_language_annotation at fe9561c
• Size: 59 evaluation samples
• Columns: anchor, positive, negative_1, and negative_2
• Approximate statistics based on the first 59 samples:

  	anchor	positive	negative_1	negative_2
  type	string	string	string	string
  details	min: 153 tokens mean: 163.86 tokens max: 172 tokens	min: 29 tokens mean: 54.46 tokens max: 148 tokens	min: 29 tokens mean: 54.85 tokens max: 133 tokens	min: 153 tokens mean: 163.68 tokens max: 172 tokens

• Samples:

  anchor	positive	negative_1	negative_2
  RPS29 ACTB PTMA TPT1 BTG1 DUSP1 TMSB10 KLF2 JUN FTH1 TMSB4X FTL ZFP36L2 HLA-DRB5 CD79A CXCR4 TSC22D3 ZNF331 MS4A1 STK17A NR4A2 RPL36A VIM HERPUD1 MAP3K8 CD69 CD83 SBDS CD52 JUNB CRIP1 STK17B VPREB3 OSER1 FOS BASP1 FNBP1 H4C3 MBP PHF20 FAM107B KLF6 SLC44A1 PAPOLA MAST3 STK4 POLR1F C12orf57 TFEB HSPE1	This measurement was conducted with 10x 5' v1. Plasmablast cells derived from the tonsil tissue of a 9-year-old female patient with recurrent tonsillitis. The cells exhibit IGH + IGK chains, with IGH being functional, in-frame, and having IGHJ602 and IGHV2-7001 genes, as well as IGKC, IGKV3-11, IGKJ2 genes with productive status, and IgG2 isotype.	This measurement was conducted with 10x 5' v1. Memory B cell derived from the tonsil tissue of a 3-year-old male human, with IGH and IGK chains, IGH functional, IGH in-frame, IGH junction length 66, IGHJ4*02, IGKC, IGKV1-27, IGKJ1, IgG1 isotype, and 50 unique molecular identifiers (UMIs) for IGK.	ACTB TMSB4X PTMA TPT1 TMSB10 FTH1 RPS29 CD52 ARPC5 VIM LGALS1 MS4A1 CD79A CMTM6 ATP5MJ S100A6 S100A4 ATP5PF DDIT4 CD82 FABP5 HLA-DRB5 CRIP1 ENO1 S100A10 NDUFB4 DEK S100A11 HCLS1 H4C3 AP2S1 EIF2S2 TRABD FTL RTRAF PSMA6 HSPE1 ZNF706 SAT1 TMCO1 ATP5PD EWSR1 SEMA4D RTF2 STK17B MRPS33 VTI1B SPCS2 SREK1IP1 CHD1
  TMSB4X RPS29 TMSB10 PTMA FTL ACTB SAT1 FTH1 MT-CO3 TSC22D3 CD52 ISG15 BTG1 OAS1 ELF1 CXCR4 TPT1 JUNB CARD16 RPL36A HERPUD1 ENO1 PAPOLA CD79A PTGES3 DUSP1 HSPD1 ATP5MJ ARPC5 STK17A ZEB2 CDC42SE1 S100A10 S100A6 MAD1L1 ICA1 CFLAR FKBP4 RBM6 LAMP2 IFRD1 JARID2 MED24 SEC62 RNF216 CD22 CLK1 ZMYND11 PHLDB1 RUNX3	This measurement was conducted with 10x 3' v2. B cell sample taken from blood tissue of a 49-year old European female with managed systemic lupus erythematosus (SLE). The cell type was enriched from peripheral blood mononuclear cells.	This measurement was conducted with 10x 3' v2. Sample is a 62-year old female of Asian ethnicity with managed systemic lupus erythematosus (SLE). The cell type of interest is a conventional dendritic cell (cDC), which was derived from peripheral blood mononuclear cells (PBMCs) obtained from blood tissue.	TMSB4X RPS29 FTL FTH1 MT-CO3 TPT1 PTMA TMSB10 CD52 ACTB CLDND1 STK4 NBEAL1 ZFP36L2 ATP5MJ DDX21 JUN S100A4 ADSL IFRD1 RTF2 SYNE2 ATP11B TRMT11 KLF6 PSME4 ENO1 TRPM3 PPIE FKBP3 NFKBIA GID8 URI1 LIPA BCCIP MTMR4 RTN4 HSPE1 PHF3 CNTRL ZNF706 CXCR4 ARL4A MT2A BTG1 IL6ST OLA1 SENP7 HNRNPD PMAIP1
  ACTB TMSB4X TMSB10 CD79A PTMA RPS29 TPT1 FTH1 MS4A1 HLA-DRB5 CD52 MARCKSL1 CD22 VPREB3 PDIA3 RPL36A MAD1L1 ARAP2 SRRT CD69 XRN1 CXCR4 SOX4 XPO4 BTG1 LARS1 TIAL1 CNOT8 ATP5MJ BASP1 NT5DC1 MRPL14 FAM133B ADSL SEC62 UBA6 ZNF800 HERPUD1 NDUFB4 FRYL STK17B GSTP1 LAT2 FTL RTRAF MRPS33 NUDC CMTM6 NFKBIA GID8	This measurement was conducted with 10x 5' v1. Naive B cell sample taken from a 5-year-old female with obstructive sleep apnea and recurrent tonsillitis, originating from the tonsil tissue.	This measurement was conducted with 10x 5' v1. Naive B cell sample taken from a 5-year-old female individual with obstructive sleep apnea and recurrent tonsillitis, isolated from tonsil tissue.	ACTB TMSB4X PTMA MS4A1 RPS29 TPT1 TMSB10 FTH1 HLA-DRB5 CD83 MYC ENO1 FTL REL DNPH1 HSPE1 HSPD1 NUDC CD79A NHP2 RFXANK DEK BTG1 ATP5PF DDX21 PCED1B HERPUD1 NFKBIA MAP3K8 PTGES3 TXNDC17 OSER1 SDCBP FABP5 NFKBID MARCKSL1 RPL36A SEC62 FAM107B CD82 SRRT GADD45B PSMA6 POLD2 SMC3 MRPL3 CCND2 HCST CD164 TMCO1

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

gene_description

• Dataset: gene_description at 338d1c3
• Size: 199 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 199 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 5.53 tokens max: 10 tokens	min: 8 tokens mean: 90.71 tokens max: 268 tokens

• Samples:

  anchor	positive
  A1BG	The protein encoded by this gene is a plasma glycoprotein of unknown function. The protein shows sequence similarity to the variable regions of some immunoglobulin supergene family member proteins. [provided by RefSeq, Jul 2008]
  A1BG-AS1	A1BG antisense RNA 1
  A1CF	Mammalian apolipoprotein B mRNA undergoes site-specific C to U deamination, which is mediated by a multi-component enzyme complex containing a minimal core composed of APOBEC-1 and a complementation factor encoded by this gene. The gene product has three non-identical RNA recognition motifs and belongs to the hnRNP R family of RNA-binding proteins. It has been proposed that this complementation factor functions as an RNA-binding subunit and docks APOBEC-1 to deaminate the upstream cytidine. Studies suggest that the protein may also be involved in other RNA editing or RNA processing events. Several transcript variants encoding a few different isoforms have been found for this gene. [provided by RefSeq, Nov 2010]

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• learning_rate: 0.05
• num_train_epochs: 1
• warmup_ratio: 0.1

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Framework Versions

• Python: 3.12.9
• Sentence Transformers: 5.0.0
• Transformers: 4.52.3
• PyTorch: 2.7.0
• Accelerate: 1.7.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}