Answers.Body: <p>You can use an autoencoder for text. For example, you can refer to this example here: <a href="https://machinelearningmastery.com/lstm-autoencoders/" rel="nofollow noreferrer">https://machinelearningmastery.com/lstm-autoencoders/</a></p>
  
  <p>For comparing the contextual similarity, you can compare the encoded vectors for the distance maybe through a formula like mean squared error. </p>
  
  <p>This works as the autoencoder compresses the input data into a vector of numbers, forcing the encoder and the decoder to learn specific features about text. The features cannot be understood by humans but have meanings. </p>
  
  <p>Another approach will be using some word or document embedding like word2vec or GloVe. IT may also work well depending on your dataset size. You need to experiment through different methods to find out which is the best.</p>
  
  <p>A supervised method can also be used if you have labels. For example, you can use a LSTM and train it like an Siamese network for LSTM using triplet loss. Here is an example: <a href="https://medium.com/@gautam.karmakar/manhattan-lstm-model-for-text-similarity-2351f80d72f1" rel="nofollow noreferrer">https://medium.com/@gautam.karmakar/manhattan-lstm-model-for-text-similarity-2351f80d72f1</a> Implementation: <a href="https://github.com/GKarmakar/deep-siamese-text-similarity" rel="nofollow noreferrer">https://github.com/GKarmakar/deep-siamese-text-similarity</a></p>
  
  <p>Hope this can help you and have a nice day</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-07T16:08:15.773Z
Answers.Text: I think you should elaborate on your answer. Why would auto-encoders (specifically, AEs + LSTM) be helpful for this task?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-07T13:41:57.167Z
Answers.LastActivityDate: 2019-11-08T03:18:25.977Z
Answers.LastEditDate: 2019-11-08T03:18:25.977Z
Answers.UserDisplayName: Clement
Body: <p>How do you build a language model to predict the contextual similarity between two documents?</p>
  
Tags: <natural-language-processing><language-model>
Title: How do you build a language model to predict the contextual similarity between two documents?
UserDisplayName: Sujeet Kumar Pandey
ThreadText: # How do you build a language model to predict the contextual similarity between two documents?
  
  (Asked by: username_0 on 2019-11-07)
  
  How do you build a language model to predict the contextual similarity between two documents?
  
  **Tags:** natural-language-processing, language-model
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-07. Score: 1
  
  You can use an autoencoder for text. For example, you can refer to this example here: <https://machinelearningmastery.com/lstm-autoencoders/>
  
  For comparing the contextual similarity, you can compare the encoded vectors for the distance maybe through a formula like mean squared error.
  
  This works as the autoencoder compresses the input data into a vector of numbers, forcing the encoder and the decoder to learn specific features about text. The features cannot be understood by humans but have meanings.
  
  Another approach will be using some word or document embedding like word2vec or GloVe. IT may also work well depending on your dataset size. You need to experiment through different methods to find out which is the best.
  
  A supervised method can also be used if you have labels. For example, you can use a LSTM and train it like an Siamese network for LSTM using triplet loss. Here is an example: <https://medium.com/@gautam.karmakar/manhattan-lstm-model-for-text-similarity-2351f80d72f1> Implementation: <https://github.com/GKarmakar/deep-siamese-text-similarity>
  
  Hope this can help you and have a nice day
  
  > **Comments:**
  > **username_2**: I think you should elaborate on your answer. Why would auto-encoders (specifically, AEs + LSTM) be helpful for this task?  
  
  ---
  
  