title: Keras/NLP Assigning weight/higher priority to position in text sequence?
body: <p>I am developing a text classifier that should be able to classify the sentiment, positive or negative in sentence. So far so good.</p>
  <p>The specificity is that I am trying to achieve is that in a text, there should be a way to assign higher weights/values for sentences starting or ending than the rest of the sentence.</p>
  <p>An example would be &quot;Today was a good day. I liked a car. I enjoyed an ice cream.&quot; The first and last sentence should contribute more to the classification than the sentence/s in the middle.</p>
  <p>So basically, a classifier should classify this as a positive sentence, but the weighting should be heavily biased to the beginning or the end.</p>
  <p>If an RNN is not suitable for this, what would be? Do I need to implement a custom loss or optimization function?</p>
  <p>I have looked into Transformer models. Some guiding advice would be appreciated.
  So far I have tried using a RNN:`</p>
  <pre><code>    max_words = 50000
  
      max_len = 80
  
      embedding_dim = 256
  
      epochs = 2
  
      batch_size = 256`
  
      model1 = Sequential()
  
      model1.add(layers.Embedding(vocab_size, embedding_dim, input_length=X_train.shape[1]))  # The embedding layer
  
      model1.add(Attention(return_sequences=True))
  
      model1.add(layers.Bidirectional(layers.LSTM(32, return_sequences=False, dropout=0.2)))
  
      model1.add(layers.Dense(64, activation='relu'))
  
      model1.add(layers.Dense(1, activation='sigmoid'))
  
      model1.compile(loss='binary_crossentropy', optimizer='adam', metrics=['binary_accuracy'])
  </code></pre>