Gpt, Transformer, Pytorch, Multi Head Attention, Decoder.
involve relationships between adjacent tokens, such as word order, part-of-speech, and basic sentence structure, without the need for a step-by-step approach. Middle attention: It encompasses the broader context of the input sequence, which may include semantic information, meanings, relationships