Gpt, Transformer, Pytorch, Multi Head Attention, Decoder.
(self_attention): MultiheadAttention( (out_proj): NonDynamicallyQuantizableLinear(in_features=2048, out_features=2048, bias=True) ) (norm1): LayerNorm((2048,), eps=1e-05, elementwise_affine=True) (dropout1): Dropout(p=0.1, inplace=False) (linear1): Linear(in_features=2048, out_features=16384,