title: Outputting batch/epoch training loss during `tf.train.MonitoredTrainingSession`
body: <p>I would like to output my loss with <code>MonitoredTrainingSession</code> every epoch or batch.
  Ideally I would love to get a flag that the epoch is ended or be able to provide a callback like in keras. I see that I can also do it by manually counting steps, but I want to use the tf functionality, which seems still poorly documented.</p>
  
  <p>From what I could find in their documentation, one can use <code>tf.train.LoggingTensorHook</code> to print the tensors every <code>n</code> steps. </p>
  
  <p>The problem however is that it prints with frequency different from what I request. When I run following with <code>every_n_iter=4</code> I get output every 2nd iteration:</p>
  
  <pre><code>tf.reset_default_graph()
  with g.as_default():
      loghook = tf.train.LoggingTensorHook([tf.reduce_mean(loss, name='m_loss')],
                                           every_n_iter=4,
                                           formatter=lambda x: "LOSS\t%.4f" % [tt for kk,tt in x.items() if kk.name.startswith('m_loss')][-1]
                                          )
      optimizer = get_optimizer(lr=lr, opt_name = opt_name)
      training_op = optimizer.minimize(loss)
      init_op = tf.global_variables_initializer()
      with tf.Session(graph=g) as sess:    
          sess.run(init_op)
      with tf.train.MonitoredTrainingSession(log_step_count_steps=1, hooks=[loghook]) as sess:
          losslist = []
          while not sess.should_stop():
              print('.')
              loss_ = sess.run(loss, feed_dict={K.learning_phase():1})
              sess.run(training_op)
              losslist.append(np.mean(loss_))
  </code></pre>
  
  <p>I am getting output like:</p>
  
  <pre><code>.
  INFO:tensorflow:LOSS    2.2416
  .
  .
  INFO:tensorflow:LOSS    2.1547
  .
  .
  INFO:tensorflow:LOSS    2.1186
  .
  .
  </code></pre>
  
  <p>etc. That is it outputs every 2nd step, not every 4th. </p>
  
  <p>The documentation says: </p>
  
  <pre><code>every_n_iter: `int`, print the values of `tensors` once every N local
        steps taken on the current worker.
  </code></pre>
  
  <p>I am running it on one local machine. Why one "local step" equals two loop python iterations? Why two and not five?</p>
  
  <p>Looking at the Python source does not seem helping. Any Google folks aware of what it is doing?</p>