Self Attention, Deep Learning, Transformers, Bert, Ml So Good.
architecture (Source: Author) But the powerful as it is, the transformer has a problem: The self-attention mechanism scales quadratically in time and memory with increasing sequence length. In other words, there is a limit to what the length of your text might be. For images, the size or resolution