title: OOM error on training, but not on validation
body: <p>A have a deep convolution network with approximate 120 layers. I attempt to train on <code>[7,368,368,1]</code> batches successfully. Also, I can easily give <code>[1,2700,2700,1]</code> as an input on validation step. But I can not increase batch size from 7 to higher (for example 20), tensorflow fails with</p>
  <pre><code> OOM when allocating tensor with shape[20,10,368,368].
  </code></pre>
  <p>How is it possible, when on validation we have <code>[1,10,2700,2700]</code> tensor?</p>