Transformer, Machine Translation, NLP, Attention Mechanism.
limitation as it holding on to information at the beginning of the sequence and encoding long-range dependencies. This problem is solved my attention mechanism by giving different weights to different words in the sentence. Attention can be explained in the figure below: In this article, I will