maybe you could go with nn.TransformerEncoder, it takes word embeddings as input, and output is learned representation, which could be fed to nn.Linear, and then nn.Softmax.
or even check out github.com/huggingface/transformers 5