Maybe that is why im not getting good results. I will try keeping the hidden state and detaching it between batches so the lstm has a notion of the previous sequences. Maybe this will solve my problem. You are a life saver! Thanks a lot!!