I have an LSTM I am having it return one-hot vectors. I just use the vocabulary size. But now its randomly generating SOS tokens and PADD tokens when I don’t want it (like in the middle of a sentence or something like that).
How are LSTM models fixed to avoid this?
Note that just removing them from the indices list causes the problem that now the first token I pass my LSTM cannot be SOS 

https://www.quora.com/unanswered/How-do-I-avoid-RNNs-LSTMS-from-producing-nonesense-tokens-like-SOS-and-PADD

  
      
      Deep Learning Course Forums – 30 Aug 19
  
  
    

How to avoid RNNs/LSTMS from producing none-sense tokens like SOS and PADD?

I have an LSTM I am having it return one-hot vectors. I just use the vocabulary size. But now its randomly generating SOS tokens and PADD tokens when I don’t want it (like in the middle of a sentence or something like that).  How are LSTM models...