Please let me elaborate a bit more.

I am working on a question-answering task. Instead of CNN, the input is from a word embedding.
So, I have 2 tensors.

An output from an LSTM (in the figure attached, this is the output from LSTM in First attention to x2_1) that holds the representations of:

1.1 the input words (X_1) and other features.
1.2 the question embedding on the input words.
Let’s say this tensor is hidden_docs. The shape of this tensor is batch_size X sequence_length X hidden_size. For example, hiddens_docs = tensor(10, 253, 768)

X_1 is the input words representation.

Let’s say this tensor is x1_emb. The shape of this tensor is batch_size X sequence_length X embedding_size. For example, x1_emb = tensor(10, 253, 300)
I need to project x1_emb on hidden_docs to get a unified representation between hidden_docs and x1_emb. And this representation then needs to be feed into x2_1 (in the ‘Attention again’).
Currently, what I am doing is since the shape is different, I am passing the x1_emb into an LSTM and able to get the shape as = batch_size X sequence_length X hidden_size.
And then I am doing a point-wise addition operation between hidden_docs and x1_emb before passing the output to x2_1. Now here I am bit confused.
Is the operation - point-wise addition between these two tensors is right operations? are there other better operations to do to capture the better representations between hidden_docs and x1_emb?