Hi again, I think that you may be right, but at the moment I don’t have a deep enough understanding to make it work. Instead, I’m thinking of trying a DNN for the system instead, the architecture for which was laid out more clearly in the paper.
I just had one question about how the output of the DNN would work.
Based on the paper, that would change my input to a  [250]  sized tensor, as I’m concatenating each one. I’m most confused about how the output would work, and how I would go about it. Do I need my final output fully connected layer to be of size 3, 5, 15, or something else completely?
The reasoning behind 3 would be that I am getting a softmax probability of there being no punctuation after the word, a comma, or a period. However, this does not account for all 5 words in the sequence. If I was to have my output be 15, would that work? In that case, could I just reshape the output to get the softmax probabilities of each label option?
If my label options are 0, 1, and 2, each meaning a type of punctuation, how do I make sure the softmax probabilities are “in the right order”, if that’s how it works?
I really appreciate all your help!