title: High GPU Memory-Usage but zero volatile gpu-util
body: <p>My new training code occupied high GPU Memory-Usage but zero volatile gpu-util</p>
  
  <p><a href="https://i.stack.imgur.com/omOcF.png" rel="noreferrer"><img src="https://i.stack.imgur.com/omOcF.png" alt="enter image description here"></a></p>
  
  <pre><code>/home/diana/data/KaggleDiabeticRetinopaI tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with     properties: 
  name: GeForce GTX 1080
  major: 6 minor: 1 memoryClockRate (GHz) 1.835
  pciBusID 0000:05:00.0
  Total memory: 7.92GiB
  Free memory: 7.81GiB
  I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0 
  I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y 
  I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlowdevice (/gpu:0) -&gt; (device: 0, name: GeForce GTX 1080, pci bus id: 0000:05:00.0)
  </code></pre>
  
  <p>I'm trying a new data generator, and the old one run with volatile gpu-util of 10%, so the environment or the tensorflow version won't be the key problem </p>
  
  <p>so I wonder if anyone could tell me what element in the code could possibly cause this problem?</p>
  
  <p>thank you very much！</p>