<p>I ended up finding <a href="https://ai.intelligentonlinetools.com/ml/document-similarity-in-machine-learning-text-analysis-with-elmo/" rel="nofollow noreferrer">this article</a> which does what I'm looking for. 
Below is the portion of code I adapted for my needs</p>



<pre><code>from sklearn.metrics.pairwise import cosine_similarity

import tensorflow_hub as hub
import tensorflow as tf

elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)

def elmo_vectors(x):
  embeddings=elmo(x, signature="default", as_dict=True)["elmo"]

  with tf.device('/device:GPU:0'):
    with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      sess.run(tf.tables_initializer())
      # return average of ELMo features
      return sess.run(tf.reduce_mean(embeddings,1))


corpus=["I'd like an apple juice",
        "An apple a day keeps the doctor away",
         "Eat apple every day",
         "We buy apples every week",
         "We use machine learning for text classification",
         "Text classification is subfield of machine learning"]


elmo_embeddings=[]
print (len(corpus))
for i in range(len(corpus)):
    print (corpus[i])
    elmo_embeddings.append(elmo_vectors([corpus[i]])[0])

print ( elmo_embeddings, len(elmo_embeddings))
print(elmo_embeddings[0].shape)
sims = cosine_similarity(elmo_embeddings, elmo_embeddings)
print(sims)
print(sims.shape)
</code></pre>

CC BY-SA 4.0
2019-08-13T21:23:31.040Z
2019-08-13T21:23:31.040Z
brokentransform
<p>I'm assuming you are trying to train a network that compares 2 sentences and give how similar they are.</p>

<p>To do that you will need the dataset (the list of sentences) and a corresponding list of 'correct answers' (the exact similarity of the sentences, which I'm assuming you don't have?).</p>

<p>Why do you need to compare them using a neural network though? For python, difflib's sequence matcher would be my suggestion, but I'm sure there are many other libraries out there :)</p>

CC BY-SA 4.0
2019-08-13T21:24:27.490Z
I appreciate your answer, but I ended up finding my solution elsewhere. Thanks again
brokentransform
CC BY-SA 4.0
2019-08-13T21:53:28.547Z
Hi Kenivia! Thanks for trying to contribute to this website. I appreciate that. However, in order to improve your answer, I would suggest you add more details to the sentence _Just write a program that does that by comparing words or something_. For example, you could mention some existing similarity metrics that could be used for this task.
nbro
CC BY-SA 4.0
2019-08-13T10:07:33.630Z
2019-08-13T23:16:12.177Z
2019-08-13T23:16:12.177Z
Kenivia
<p>Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can <a href="https://allennlp.org/elmo" rel="nofollow noreferrer">ELMO embeddings</a> be used to train a model that can give you the <span class="math-container">$n$</span> most similar sentences to an input string?</p>

<p>For reference, gensim provides a <a href="https://radimrehurek.com/gensim/models/doc2vec.html" rel="nofollow noreferrer">doc2vec</a> model that can be trained on a list of strings, then you can use the trained model to infer a vector from some input string. That inferred vector can then be used to find the <span class="math-container">$n$</span> most similar vectors.</p>

<p>Could something similar be done, but using ELMO embedding instead?</p>

<p>Any guidance would be greatly appreciated. </p>

<natural-language-processing><word-embedding>
Can ELMO embeddings be used to find the n most similar sentences?
brokentransform
# Can ELMO embeddings be used to find the n most similar sentences?

(Asked by: username_0 on 2019-08-12)

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can [ELMO embeddings](https://allennlp.org/elmo) be used to train a model that can give you the $n$ most similar sentences to an input string?

For reference, gensim provides a [doc2vec](https://radimrehurek.com/gensim/models/doc2vec.html) model that can be trained on a list of strings, then you can use the trained model to infer a vector from some input string. That inferred vector can then be used to find the $n$ most similar vectors.

Could something similar be done, but using ELMO embedding instead?

Any guidance would be greatly appreciated.

**Tags:** natural-language-processing, word-embedding


---

## 2 answers

### Answer by username_0 on 2019-08-13. Score: 3

I ended up finding this article which does what I'm looking for. 
Below is the portion of code I adapted for my needs
from sklearn.metrics.pairwise import cosine_similarity

import tensorflow_hub as hub
import tensorflow as tf

elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)

def elmo_vectors(x):
  embeddings=elmo(x, signature="default", as_dict=True)["elmo"]

  with tf.device('/device:GPU:0'):
    with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      sess.run(tf.tables_initializer())
      # return average of ELMo features
      return sess.run(tf.reduce_mean(embeddings,1))


corpus=["I'd like an apple juice",
        "An apple a day keeps the doctor away",
         "Eat apple every day",
         "We buy apples every week",
         "We use machine learning for text classification",
         "Text classification is subfield of machine learning"]


elmo_embeddings=[]
print (len(corpus))
for i in range(len(corpus)):
    print (corpus[i])
    elmo_embeddings.append(elmo_vectors([corpus[i]])[0])

print ( elmo_embeddings, len(elmo_embeddings))
print(elmo_embeddings[0].shape)
sims = cosine_similarity(elmo_embeddings, elmo_embeddings)
print(sims)
print(sims.shape)


---

### Answer by username_1 on 2019-08-13. Score: -1

I'm assuming you are trying to train a network that compares 2 sentences and give how similar they are.

To do that you will need the dataset (the list of sentences) and a corresponding list of 'correct answers' (the exact similarity of the sentences, which I'm assuming you don't have?).

Why do you need to compare them using a neural network though? For python, difflib's sequence matcher would be my suggestion, but I'm sure there are many other libraries out there :)

> **Comments:**
> **username_0**: I appreciate your answer, but I ended up finding my solution elsewhere. Thanks again  
> **username_2**: Hi username_1! Thanks for trying to contribute to this website. I appreciate that. However, in order to improve your answer, I would suggest you add more details to the sentence _Just write a program that does that by comparing words or something_. For example, you could mention some existing similarity metrics that could be used for this task.  

---

