AI, Large Language Models, Transformers, Machine Learning, Getting Started.
There are many. But the two mechanisms that are truly the force behind the transformers are attention weighting and feed-forward networks (FFN). What is attention-weighting? Attention-weighting is a technique by which the model learns which part of the incoming sequence needs to be focused on.