Why are you multiplying the hidden states and the output of the lstm? What do you mean by a linear decoder? Could you site the source of this architecture? The standard attention mechanism does not look like this.