I am trying to do NN based on this article http://nlp.seas.harvard.edu/2018/04/03/attention.html 1
Or based on the standard nn.transformer module
I have numbers instead of words. The size of one sequence is 5. At the output(I have three neurons), I need to predict three values ​​(0, 1, 2). That is, at each step I take a window of size 5 and predict 0 or 1 or 2. In fact, this is a classification into 3 classes. Can I use the standard nn.transformer module and use my data instead of words? Or what do you advise me to do?

transformer.png972×616 6.53 KB