Technology, Artificial Intelligence, Machine Learning, Data Science, Deep Thoughts.
LSTM (BN-LSTM) cell and the Transformer Network for its Multi-Head Attention Mechanism. The Multi-Head Attention Mechanism is used inside the cell such that it can query its own k past values with the attention window. This has the effect of augmenting the rank of the tensor with the attention