title: when should i initialize state in LSTM code?
body: <p>This is the LSTM code from UDACITY for sentiment classification.</p>
  
  <p>Here is the link of whole sentence-rnn code: <a href="https://github.com/udacity/deep-learning/blob/ed675e0b7089316fb14f45845d5b088b05daae11/sentiment-rnn/Sentiment_RNN_Solution.ipynb" rel="nofollow noreferrer">udacity/sentiment-rnn</a></p>
  
  <p>I wonder why they initialize the cell state right under the for loop for epoch.</p>
  
  <p>I think the cell state must be zero-initialize when the input sentence changes, so it must be under the mini-batch for loop statement.</p>
  
  <pre><code>## part of the sentence-rnn code
  # Getting an initial state of all zeros
  initial_state = cell.zero_state(batch_size, tf.float32)
  
  with tf.Session(graph=graph) as sess:
      sess.run(tf.global_variables_initializer())
      iteration = 1
      for e in range(epochs):
          state = sess.run(initial_state)    ###### i think this line
  
          for ii, (x, y) in enumerate(get_batches(train_x, train_y, batch_size), 1):
              ###### should be here
              feed = {inputs_: x,
                      labels_: y[:, None],
                      keep_prob: 0.5,
                      initial_state: state}
              loss, state, _ = sess.run([cost, final_state, optimizer], feed_dict=feed)
  </code></pre>
  
  <p>anyone who can explain why?</p>
  
  <p>Thanks!</p>