Self Attention, Deep Learning, Transformers, Bert, Ml So Good.
along with residual connections and layer normalizations. This had been proven to be extremely successful on multiple data modalities. It started with text, became mainstream on both image and text, then on the image only and then it moved on to speech and even further than that. Transformer