Gpt, Transformer, Pytorch, Multi Head Attention, Decoder.
bias=True) (linear2): Linear(in_features=16384, out_features=2048, bias=True) (norm2): LayerNorm((2048,), eps=1e-05, elementwise_affine=True) (dropout2): Dropout(p=0.1, inplace=False) ) ) (linear): Linear(in_features=2048, out_features=10000, bias=True) (softmax): LogSoftmax(dim=-1) ) Now, you have