Thank you! I’m looking through your notebook and will make some changes to my approach (e.g. sample text randomly). I think it’ll be helpful.
What I did regarding top choices is that I defined a threshold (e.g. 0.8 times the max), and sampled randomly between all characters that exceeded the threshold. But since the network just ends up repeating the last words of the input, I think it trains to a point where the gap between 1st choice and 2nd choice is super large…
I have a feeling my problem may have to do with the way I’m feeding the network the input and target during training or something? I’m having a hard time wrapping my head around what, conceptually, can lead to this repetition of the last words of the input…?