How can I get similarity matching ?
## ❓ Questions & Help
Is there any way that can help me calculate the similarity between 2 questions ? Sometimes the questions is out of the scope of the data set questions. Using a simple similarity algorithm will always return the most similar even if it is not really correct. 
It is the same thing as here : 
https://github.com/deepmipt/dp_notebooks/blob/master/DP_BERT.ipynb

from deeppavlov import build_model, configs
model = build_model(configs.squad.squad_bert, download=True)
model(['DeepPavlov is a library for NLP and dialogue systems.'], ['What is DeepPavlov?'])

<!-- A clear and concise description of the question. -->

Not sure to understand what you mean by `Using a simple similarity algorithm will always return the most similar even if it is not really correct`. What kind of simple similarity algo are you evoking here? What do you mean those simple algorithms aren't precise enough for your usecase?

Considering sentence similarity algorithms, I know:
- Statistical approach using bag-of-words TF-IDF-based methods like BM25 (better on longer docs than sentences).
- bag-of-word sentences and pooling (average-like) on word embeddings (word2vec-like) weighted by TF-IDF on a corpus for example.
- Full Sentence Embedding learnt directly on a similarity training set (maybe finetuned on your domain). It builds sentence embedding in a vector-space in which you can compute distance between sentences. Those models often uses siamese approach based on pre-trained language models such as BERT.

Sentence embedding technique requires more work and domain knowledge but is the one reaching the highest metrics in SOTA. Other techniques can be enough depending on your needs and domain.

<|||||>Excuse me for not being clear enough, 
I wanted to say that if I have an FAQ data set and I want to get the most similar question for the user's question then, cosine similarity or TF-IDF will always give back a question even if it is not related. 
This is about classifying either that question does have a similar one or not like does the snippet of code I posted earlier from deeppavlov. <|||||>No need to excuse ;)
Except an approach based on a dataset classifying "similar and non-similar" on your domain, any approach based on a score will require that you set a threshold to discriminate similar and non-similar. It can be fuzzy but sentence similarity is a very relative concept in any case.


<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
