<p>I think you should use Keras embedding layer. It will be too easier than what you are doing.</p>

<h3>Steps</h3>

<ul>
<li>Create Embedding Matrix</li>
<li>add matrix to embedding layer while building model.</li>
</ul>

<h3>You will find detailed article</h3>

<p><a href="https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html" rel="nofollow noreferrer">https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html</a></p>

CC BY-SA 4.0
2020-05-21T10:57:07.100Z
2020-05-21T10:57:07.100Z
PSKP
<p>I want to use a pretrained model found in [BERT Embeddings] <a href="https://github.com/UKPLab/sentence-transformers" rel="nofollow noreferrer">https://github.com/UKPLab/sentence-transformers</a> and I want to add a layer to get the sentence embeddings from the model and pass on to the next layer. How do I approach this?</p>

<p>The inputs would be an array of documents and each document containing an array of sentences.</p>

<p>The input to the model itself is a list of sentences where it will return a list of embeddings. </p>

<p>This is what I've tried but couldn't solve the errors:</p>

<pre><code>def get_embeddings(input_data):

    input_embed = []
    for doc in input_data:
      doc = tf.unstack(doc)
      doc_arr = asarray(doc)
      doc = [el.decode('UTF-8') for el in doc_arr]
      doc = list(doc)
      assert(type(doc)== list)

      new_doc = []
      for sent in doc:
        sent = tf.unstack(sent)
        new_doc.append(str(sent))
        assert(type(sent)== str)

      embedding= model.encode(new_doc)  # Accepts lists of strings to return BERT sentence embeddings
      input_embed.append(np.array(embedding))

    return tf.convert_to_tensor(input_embed, dtype=float)


sentences = tf.keras.layers.Input(shape=(3,5)) #test shape
sent_embed = tf.keras.layers.Lambda(get_embeddings)


x = sent_embed(sentences)
</code></pre>

<p><a href="https://i.sstatic.net/1O8sY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1O8sY.png" alt="Errors"></a></p>

<neural-networks><word-embedding><bert><text-summarization><pretrained-models>
How to add a pretrained model to my layers to get embeddings?
inquisitive
# How to add a pretrained model to my layers to get embeddings?

(Asked by: username_0 on 2020-05-20)

I want to use a pretrained model found in [BERT Embeddings] https://github.com/UKPLab/sentence-transformers and I want to add a layer to get the sentence embeddings from the model and pass on to the next layer. How do I approach this?
The inputs would be an array of documents and each document containing an array of sentences.
The input to the model itself is a list of sentences where it will return a list of embeddings. 
This is what I've tried but couldn't solve the errors:
def get_embeddings(input_data):

    input_embed = []
    for doc in input_data:
      doc = tf.unstack(doc)
      doc_arr = asarray(doc)
      doc = [el.decode('UTF-8') for el in doc_arr]
      doc = list(doc)
      assert(type(doc)== list)

      new_doc = []
      for sent in doc:
        sent = tf.unstack(sent)
        new_doc.append(str(sent))
        assert(type(sent)== str)

      embedding= model.encode(new_doc)  # Accepts lists of strings to return BERT sentence embeddings
      input_embed.append(np.array(embedding))

    return tf.convert_to_tensor(input_embed, dtype=float)


sentences = tf.keras.layers.Input(shape=(3,5)) #test shape
sent_embed = tf.keras.layers.Lambda(get_embeddings)


x = sent_embed(sentences)

**Tags:** neural-networks, word-embedding, bert, text-summarization, pretrained-models


---

## 1 answers

### Answer by username_1 on 2020-05-21. Score: 1

I think you should use Keras embedding layer. It will be too easier than what you are doing.

### Steps

* Create Embedding Matrix
* add matrix to embedding layer while building model.

### You will find detailed article

<https://www.cs.uaf.edu/2011/spring/cs641/lecture/04_05_modeling.html>


---

