readme: ---
  language:
  - de
  - en
  - multilingual
  license: mit
  tags:
  - sentence_embedding
  - search
  - pytorch
  - xlm-roberta
  - roberta
  - xlm-r-distilroberta-base-paraphrase-v1
  - paraphrase
  datasets:
  - stsb_multi_mt
  metrics:
  - Spearman’s rank correlation
  - cosine similarity
  ---
  
  # Cross English & German RoBERTa for Sentence Embeddings
  This model is intended to [compute sentence (text) embeddings](https://www.sbert.net/examples/applications/computing-embeddings/README.html) for English and German text. These embeddings can then be compared with [cosine-similarity](https://en.wikipedia.org/wiki/Cosine_similarity) to find sentences with a similar semantic meaning. For example this can be useful for [semantic textual similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html), [semantic search](https://www.sbert.net/docs/usage/semantic_search.html), or [paraphrase mining](https://www.sbert.net/docs/usage/paraphrase_mining.html). To do this you have to use the [Sentence Transformers Python framework](https://github.com/UKPLab/sentence-transformers).
  
  The speciality of this model is that it also works cross-lingually. Regardless of the language, the sentences are translated into very similar vectors according to their semantics. This means that you can, for example, enter a search in German and find results according to the semantics in German and also in English. Using a xlm model and _multilingual finetuning with language-crossing_ we reach performance that even exceeds the best current dedicated English large model (see Evaluation section below).
  
  > Sentence-BERT (SBERT) is a  modification  of  the  pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT.
  
  Source: [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084)
  
  This model is fine-tuned from [Philip May](https://may.la/) and open-sourced by [T-Systems-onsite](https://www.t-systems-onsite.de/). Special thanks to [Nils Reimers](https://www.nils-reimers.de/) for your awesome open-source work, the Sentence Transformers, the models and your help on GitHub.
  
  ## How to use
  To use this model install the `sentence-transformers` package (see here: <https://github.com/UKPLab/sentence-transformers>).
  
  ```python
  from sentence_transformers import SentenceTransformer
  model = SentenceTransformer('T-Systems-onsite/cross-en-de-roberta-sentence-transformer')
  ```
  
  For details of usage and examples see here:
  - [Computing Sentence Embeddings](https://www.sbert.net/docs/usage/computing_sentence_embeddings.html)
  - [Semantic Textual Similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html)
  - [Paraphrase Mining](https://www.sbert.net/docs/usage/paraphrase_mining.html)
  - [Semantic Search](https://www.sbert.net/docs/usage/semantic_search.html)
  - [Cross-Encoders](https://www.sbert.net/docs/usage/cross-encoder.html)
  - [Examples on GitHub](https://github.com/UKPLab/sentence-transformers/tree/master/examples)
  
  ## Training
  The base model is [xlm-roberta-base](https://huggingface.co/xlm-roberta-base). This model has been further trained by [Nils Reimers](https://www.nils-reimers.de/) on a large scale paraphrase dataset for 50+ languages. [Nils Reimers](https://www.nils-reimers.de/) about this [on GitHub](https://github.com/UKPLab/sentence-transformers/issues/509#issuecomment-712243280):
  
  >A paper is upcoming for the paraphrase models.
  >
  >These models were trained on various datasets with Millions of examples for paraphrases, mainly derived from Wikipedia edit logs, paraphrases mined from Wikipedia and SimpleWiki, paraphrases from news reports, AllNLI-entailment pairs with in-batch-negative loss etc.
  >
  >In internal tests, they perform much better than the NLI+STSb models as they have see more and broader type of training data. NLI+STSb has the issue that they are rather narrow in their domain and do not contain any domain specific words / sentences (like from chemistry, computer science, math etc.). The paraphrase models has seen plenty of sentences from various domains.
  >
  >More details with the setup, all the datasets, and a wider evaluation will follow soon.
  
  The resulting model called `xlm-r-distilroberta-base-paraphrase-v1` has been released here: <https://github.com/UKPLab/sentence-transformers/releases/tag/v0.3.8>
  
  Building on this cross language model we fine-tuned it for English and German language on the [STSbenchmark](http://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark) dataset. For German language we used the dataset of our [German STSbenchmark dataset](https://github.com/t-systems-on-site-services-gmbh/german-STSbenchmark) which has been translated with [deepl.com](https://www.deepl.com/translator). Additionally to the German and En