Have you checked the PyTorch 12 tutorial on the Transformer models? You can see the last layer is a linear layer with output size of the number of tokens, because it is a LM model, in your case the output will be the text class of the input. I would just base your work on that guide as it is really nicely written.