title: CUDA Profiler: Calculate memory and compute utilization
body: <p>I am trying to establish two overall measurements for memory bandwidth utilization and compute throughput utilization for my GPU-accelerated application using CUDA nsight profiler on ubuntu. The application runs on a Tesla K20c GPU.</p>
  
  <p>The two measurements I want are to some extend comparable to the ones given in this graph:
  <img src="https://i.stack.imgur.com/FV1Kr.png" alt="Figure one"></p>
  
  <p>The problems are that no exact numbers are given here and more importantly that I do not know how these percentages are being calculated.</p>
  
  <p><strong>Memory Bandwidth Utilization</strong></p>
  
  <p>The Profiler tells me that my GPU has a Max Global Memory Bandwidth of 208 GB/s.</p>
  
  <p><img src="https://i.stack.imgur.com/GV4aE.png" alt="enter image description here"></p>
  
  <p>Does this refer to Device Memory BW or the Global Memory BW? It sais Global but the first one makes more sense to me.</p>
  
  <p>For my kernel the profiler tells me that the Device Memory Bandwidth is 98.069 GB/s.</p>
  
  <p><img src="https://i.stack.imgur.com/akOfz.png" alt="enter image description here"></p>
  
  <p>Assuming that the max of 208 GB/s refer to the Device Memory could I then simply calculate the Memory BW Utilization as 90.069/208 = 43%? Note that this kernel is executed multiple times without additional CPU-GPU data transfers. The system BW is therefore not important.</p>
  
  <p><strong>Compute Throughput Utilization</strong></p>
  
  <p>I am not exactly sure what the best way is to put Compute Throughput Utilization into a number. My best guess is to use the Instructions per Cycle to max Instructions per cycle ratio. The profiler tells me that the max IPC is 7 (see picture above). </p>
  
  <p>First of all, what does that actually mean? Each multiprocessor has 192 cores and therefore a maximum of 6 active warps. Wouldnt that mean that max IPC should be 6?</p>
  
  <p>The profiler tells me that my kernel has issued IPC = 1.144 and executed IPC = 0.907. Should I calculate the compute utilization as 1.144/7 = 16% or 0.907/7 = 13% or none of these?</p>
  
  <p>Are these two measurements (Memory and compute utilization) giving an adequate first impression of how efficiently my kernel is using the resources? Or are there other important metrics that should be included?</p>
  
  <p><strong>Additional Graph</strong></p>
  
  <p><img src="https://i.stack.imgur.com/oe7zF.png" alt="enter image description here"></p>