title: Java G1 GC tuning issue: old regions not collected
body: <p>I am currently trying to tune Apache NiFi to get it work with a high throughput flow, but I cannot avoid Full GCs.</p>
  
  <p>When the flow is started, really quick young GCs occur, but they are not able to cope with the allocation demand until eventually the full GC is triggered. This situation happens with different heap sizes (from 8GB to 50GB) and basic configuration (only region size and designated threads have been configured according to the <a href="http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html" rel="nofollow noreferrer">oracle documentation</a>):</p>
  
  <pre><code>-XX:ConcGCThreads=3
  -XX:G1HeapRegionSize=16
  -XX:InitialHeapSize=34359738368
  -XX:MaxHeapSize=34359738368
  -XX:ParallelGCThreads=10
  -XX:+ParallelRefProcEnabled
  -XX:+PrintAdaptiveSizePolicy
  -XX:+PrintGC
  -XX:+PrintGCDateStamps
  -XX:+PrintGCDetails
  -XX:+PrintGCTimeStamps
  -XX:+UnlockExperimentalVMOptions
  -XX:+UseG1GC
  </code></pre>
  
  <p>The following is the GC dump analysis performed using GCViewer:</p>
  
  <p><a href="https://i.stack.imgur.com/Fs6Lz.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Fs6Lz.png" alt="enter image description here"></a></p>
  
  <p>Looking at the logs, I've noticed that the count of old regions selected for mixed GCs is always zero. According to this <a href="https://www.infoq.com/articles/tuning-tips-G1-GC" rel="nofollow noreferrer">really useful article</a>, you can increase that dropping the <strong>InitiatingHeapOccupancyPercent</strong>, and allowing the marking phase to start earlier. This does not seem to have any effect, given that the count of old regions selected remains zero:</p>
  
  <pre><code>[G1Ergonomics (CSet Construction) finish choosing CSet, eden: 1777 regions, survivors: 251 regions, old: 0 regions, predicted pause time: 216.28 ms, target pause time: 200.00 ms]
  </code></pre>
  
  <p>According to the oracle documentation, there are experimental flags I could use, like <strong>G1MixedGCLiveThresholdPercent=65</strong>, but even if I add the <strong>UnlockExperimentalVMOptions</strong> flag before everything else, I get the following error:</p>
  
  <pre><code>2016-10-10 14:48:14,285 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: VM option 'G1MixedGCLiveThresholdPercent' is experimental and must be enabled via -XX:+UnlockExperimentalVMOptions
  </code></pre>
  
  <p>Basically, the flag is ignored.</p>
  
  <p>In first instance, could it be the the non-collection of the old regions to trigger the full GCs? If so, how can I manage to get the old regions collected and free enough memory for the application need?</p>
  
  <p>Thank you all for your help.</p>