Transformers, Deep Learning, Machine Learning, Editors Pick, Data Science.
the key difference is that Decoder architectures employ a masked (or causal) self-attention layer, so that the attention mechanism is only able to attend to the current and previous elements of the input sequence; this means that the contextual embedding produced only considers the previous