readme: ---
  pipeline_tag: sentence-similarity
  tags:
  - sentence-transformers
  - feature-extraction
  - sentence-similarity
  - transformers
  widget:
  - source_sentence: "This is a Norwegian boy"
    sentences:
      - "Dette er en norsk gutt"
      - "This is an English boy"
      - "This is a dog"
    example_title: "Cross Language"
  - source_sentence: "Det er noen dyr utenfor vinduet"
    sentences:
      - "På utsiden kan jeg høre noen hunder"
      - "Noen mennesker prater utenfor vinduet"
      - "Alle burde ha kjæledyr"
    example_title: "Paraphrases"
  - source_sentence: "En kvinne sitter i en stol"
    sentences:
      - "A woman is sitting in a chair"
      - "Hun slapper av og leser i en bok"
      - "Hun løper maraton"
    example_title: "Paraphrases across language"
    
  ---
  
  # NB-SBERT-BASE
  NB-SBERT-BASE is a [SentenceTransformers](https://www.SBERT.net) model trained on a [machine translated version of the MNLI dataset](https://huggingface.co/datasets/NbAiLab/mnli-norwegian), starting from [nb-bert-base](https://huggingface.co/NbAiLab/nb-bert-base). 
  
  The model maps sentences & paragraphs to a 768 dimensional dense vector space. This vector can be used for tasks like clustering and semantic search. Below we give some examples on how to use the model. The easiest way is to simply measure the cosine distance between two sentences. Sentences that are close to each other in meaning, will have a small cosine distance and a similarity close to 1. The model is trained in such a way that similar sentences in different languages should also be close to each other. Ideally, an English-Norwegian sentence pair should have high similarity.
  
  ## Embeddings and Sentence Similarity (Sentence-Transformers)
  
  As seen above, using the library [sentence-transformers](https://www.SBERT.net) makes the use of these models quite convenient:
  
  ```bash
  pip install -U sentence-transformers
  ```
  
  Then you can use the model like this:
  
  ```python
  from sentence_transformers import SentenceTransformer, util
  sentences = ["This is a Norwegian boy", "Dette er en norsk gutt"]
  
  model = SentenceTransformer('NbAiLab/nb-sbert-base')
  embeddings = model.encode(sentences)
  print(embeddings)
  
  # Compute cosine-similarities with sentence transformers
  cosine_scores = util.cos_sim(embeddings[0],embeddings[1])
  print(cosine_scores)
  
  # Compute cosine-similarities with SciPy
  from scipy import spatial
  scipy_cosine_scores = 1 - spatial.distance.cosine(embeddings[0],embeddings[1])
  print(scipy_cosine_scores)
  
  # Both should give 0.8250 in the example above.
  
  ```
  
  
  ## Embeddings and Sentence Similarity (HuggingFace Transformers)
  Without [sentence-transformers](https://www.SBERT.net), you can still use the model. First, you pass in your input through the transformer model, then you have to apply the right pooling-operation on top of the contextualized word embeddings.
  
  ```python
  from transformers import AutoTokenizer, AutoModel
  import torch
  
  
  #Mean Pooling - Take attention mask into account for correct averaging
  def mean_pooling(model_output, attention_mask):
      token_embeddings = model_output[0] #First element of model_output contains all token embeddings
      input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
      return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)
  
  
  # Sentences we want sentence embeddings for
  sentences = ["This is a Norwegian boy", "Dette er en norsk gutt"]
  
  # Load model from HuggingFace Hub
  tokenizer = AutoTokenizer.from_pretrained('NbAiLab/nb-sbert-base')
  model = AutoModel.from_pretrained('NbAiLab/nb-sbert-base')
  
  # Tokenize sentences
  encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
  
  # Compute token embeddings
  with torch.no_grad():
      model_output = model(**encoded_input)
  
  # Perform pooling. In this case, mean pooling.
  embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
  
  print(embeddings)
  
  # Compute cosine-similarities with SciPy
  from scipy import spatial
  scipy_cosine_scores = 1 - spatial.distance.cosine(embeddings[0],embeddings[1])
  print(scipy_cosine_scores)
  
  # This should give 0.8250 in the example above.
  
  ```
  ## SetFit - Few Shot Classification
  [SetFit](https://github.com/huggingface/setfit) is a method for using sentence-transformers to solve one of major problem that all NLP researchers are facing: Too few labeled training examples. The 'nb-sbert-base' can be plugged directly into the SetFit library. Please see [this tutorial](https://huggingface.co/blog/setfit) for how to use this technique.
  
  
  ## Keyword Extraction
  The model can be used for extracting keywords from text. The basic technique is to find the words that are most similar to the document. There are various frameworks for doing this. An easy way is to use [KeyBERT](https://github.com/MaartenGr/KeyBERT). This example shows how this can be done.
  
  ```bash
  pip install keybert
  ```
  
  ```python
  from keybert import KeyBERT
  from sentence_transformers import SentenceTran