Alexander0:

inputs


It depends on the context and there are many ways to computer attention.

One of the ways, For language translation where you have an encoder and a decoder
Here, you can compute the softmax of the dot product of the Encoder and Decoder hidden state. Then multiple of the softmax result with the Encoder hidden state the get a context vector to input into the Decoder for prediction.

In your case, you only have an encoder. In this case you can use out1 and h_n as input for attention.
I did participated in a kaggle project where I faced similar problem and I used both the Output and hidden of LSTM  for attention and it worked. You can check implementation here
  
      
      github.com
  
  
    

abdulsalam-bande/Kaggle_Projects

master/jigsaw-toxic-comment-classification-challenge

  This repository contains Kaggle projects I participated in.