I don’t have much experience with LSTM’s but the problem could be either the learning rate was too high or the fact that you intialized your network with an constant (zeros) so your network won’t backpropagate through it. Initialize it with random numbers.