Thank you so much! I was able to solve it.
I only got one more question. Currently, my model only works with a fix sequene and batchsize. But when I want to predict values, I will sometimes have a batch_size = 1. Since I am initializing the LSTM with the batch_size, likes this:
model = Combine(input_dim, hidden_dim, layer_dim, output_dim)

where hidden_dim corresponds to the batch_size, I get an error when using a different batch_size since I use the hidden_dim when initializing the hidden and cell state. Is there a way to make this variable so that it works for variable batch_sizes?