Answers.Body: <p>I think you should use Keras embedding layer. It will be too easier than what you are doing.</p>
  
  <h3>Steps</h3>
  
  <ul>
  <li>Create Embedding Matrix</li>
  <li>add matrix to embedding layer while building model.</li>
  </ul>
  
  <h3>You will find detailed article</h3>
  
  <p><a href="https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html" rel="nofollow noreferrer">https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-21T10:57:07.100Z
Answers.LastActivityDate: 2020-05-21T10:57:07.100Z
Answers.UserDisplayName: PSKP
Body: <p>I want to use a pretrained model found in [BERT Embeddings] <a href="https://github.com/UKPLab/sentence-transformers" rel="nofollow noreferrer">https://github.com/UKPLab/sentence-transformers</a> and I want to add a layer to get the sentence embeddings from the model and pass on to the next layer. How do I approach this?</p>
  
  <p>The inputs would be an array of documents and each document containing an array of sentences.</p>
  
  <p>The input to the model itself is a list of sentences where it will return a list of embeddings. </p>
  
  <p>This is what I've tried but couldn't solve the errors:</p>
  
  <pre><code>def get_embeddings(input_data):
  
      input_embed = []
      for doc in input_data:
        doc = tf.unstack(doc)
        doc_arr = asarray(doc)
        doc = [el.decode('UTF-8') for el in doc_arr]
        doc = list(doc)
        assert(type(doc)== list)
  
        new_doc = []
        for sent in doc:
          sent = tf.unstack(sent)
          new_doc.append(str(sent))
          assert(type(sent)== str)
  
        embedding= model.encode(new_doc)  # Accepts lists of strings to return BERT sentence embeddings
        input_embed.append(np.array(embedding))
  
      return tf.convert_to_tensor(input_embed, dtype=float)
  
  
  sentences = tf.keras.layers.Input(shape=(3,5)) #test shape
  sent_embed = tf.keras.layers.Lambda(get_embeddings)
  
  
  x = sent_embed(sentences)
  </code></pre>
  
  <p><a href="https://i.sstatic.net/1O8sY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1O8sY.png" alt="Errors"></a></p>
  
Tags: <neural-networks><word-embedding><bert><text-summarization><pretrained-models>
Title: How to add a pretrained model to my layers to get embeddings?
UserDisplayName: inquisitive
ThreadText: # How to add a pretrained model to my layers to get embeddings?
  
  (Asked by: username_0 on 2020-05-20)
  
  I want to use a pretrained model found in [BERT Embeddings] https://github.com/UKPLab/sentence-transformers and I want to add a layer to get the sentence embeddings from the model and pass on to the next layer. How do I approach this?
  The inputs would be an array of documents and each document containing an array of sentences.
  The input to the model itself is a list of sentences where it will return a list of embeddings. 
  This is what I've tried but couldn't solve the errors:
  def get_embeddings(input_data):
  
      input_embed = []
      for doc in input_data:
        doc = tf.unstack(doc)
        doc_arr = asarray(doc)
        doc = [el.decode('UTF-8') for el in doc_arr]
        doc = list(doc)
        assert(type(doc)== list)
  
        new_doc = []
        for sent in doc:
          sent = tf.unstack(sent)
          new_doc.append(str(sent))
          assert(type(sent)== str)
  
        embedding= model.encode(new_doc)  # Accepts lists of strings to return BERT sentence embeddings
        input_embed.append(np.array(embedding))
  
      return tf.convert_to_tensor(input_embed, dtype=float)
  
  
  sentences = tf.keras.layers.Input(shape=(3,5)) #test shape
  sent_embed = tf.keras.layers.Lambda(get_embeddings)
  
  
  x = sent_embed(sentences)
  
  **Tags:** neural-networks, word-embedding, bert, text-summarization, pretrained-models
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-21. Score: 1
  
  I think you should use Keras embedding layer. It will be too easier than what you are doing.
  
  ### Steps
  
  * Create Embedding Matrix
  * add matrix to embedding layer while building model.
  
  ### You will find detailed article
  
  <https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html>
  
  
  ---
  
  