title: How to print out values of LSTM gates in TensorFlow?
body: <p>I'm using a TensorFlow LSTM for a language model (I have a sequence of words and want to predict the next word), and as I'm running the language model, I want to print out the values of the forget, input, transform, and and output gates at each step. How do I do this?</p>
  
  <p>From inspecting the code in <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/rnn/python/ops/rnn_cell.py" rel="noreferrer">https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/rnn/python/ops/rnn_cell.py</a>, I see that the <code>LayerNormBasicLSTMCell</code> class has a <code>call</code> method that contains the <code>i, j, f, o</code> variables that I want to print out.</p>
  
  <pre><code>  def call(self, inputs, state):
      """LSTM cell with layer normalization and recurrent dropout."""
      c, h = state
      args = array_ops.concat([inputs, h], 1)
      concat = self._linear(args)
  
      i, j, f, o = array_ops.split(value=concat, num_or_size_splits=4, axis=1)
      if self._layer_norm:
        i = self._norm(i, "input")
        j = self._norm(j, "transform")
        f = self._norm(f, "forget")
        o = self._norm(o, "output")
  
      g = self._activation(j)
      if (not isinstance(self._keep_prob, float)) or self._keep_prob &lt; 1:
        g = nn_ops.dropout(g, self._keep_prob, seed=self._seed)
  
      new_c = (c * math_ops.sigmoid(f + self._forget_bias)
               + math_ops.sigmoid(i) * g)
      if self._layer_norm:
        new_c = self._norm(new_c, "state")
      new_h = self._activation(new_c) * math_ops.sigmoid(o)
  
      new_state = core_rnn_cell.LSTMStateTuple(new_c, new_h)
      return new_h, new_state
  </code></pre>
  
  <p>However, is there an easy way for me to print these variables out? Or do I have to basically recreate the relevant lines of code in this method, in my script where I'm running the LTSM?</p>