Solved by dav-ell in post #8 
              
              
                Ah, perfect! Yep, this makes sense. Thanks very much for explaining it. 
I also want to offer a revision to my previous post. Turns out you get very bad model performance if you do that training sub-loop like I was. The correct way to do teacher forcing is just to pass the targets shifted left one. …