title: IOException: Too many open files
body: <p>I'm trying to debug a file descriptor leak in a Java webapp running in Jetty 7.0.1 on Linux.</p>
  
  <p>The app had been happily running for a month or so when requests started to fail due to <em>too many open files</em>, and Jetty had to be restarted.</p>
  
  <pre><code>java.io.IOException: Cannot run program [external program]: java.io.IOException: error=24, Too many open files
      at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
      at java.lang.Runtime.exec(Runtime.java:593)
      at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58)
      at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:246)
  </code></pre>
  
  <p>At first I thought the issue was with the code that launches the external program, but it's using <a href="http://commons.apache.org/exec/" rel="noreferrer">commons-exec</a> and I don't see anything wrong with it:</p>
  
  <pre><code>CommandLine command = new CommandLine("/path/to/command")
      .addArgument("...");
  ByteArrayOutputStream errorBuffer = new ByteArrayOutputStream();
  Executor executor = new DefaultExecutor();
  executor.setWatchdog(new ExecuteWatchdog(PROCESS_TIMEOUT));
  executor.setStreamHandler(new PumpStreamHandler(null, errorBuffer));
  try {
      executor.execute(command);
  } catch (ExecuteException executeException) {
      if (executeException.getExitValue() == EXIT_CODE_TIMEOUT) {
          throw new MyCommandException("timeout");
      } else {
          throw new MyCommandException(errorBuffer.toString("UTF-8"));
      }
  }
  </code></pre>
  
  <p>Listing open files on the server I can see a high number of FIFOs:</p>
  
  <pre><code># lsof -u jetty
  ...
  java    524 jetty  218w  FIFO        0,6      0t0 19404236 pipe
  java    524 jetty  219r  FIFO        0,6      0t0 19404008 pipe
  java    524 jetty  220r  FIFO        0,6      0t0 19404237 pipe
  java    524 jetty  222r  FIFO        0,6      0t0 19404238 pipe
  </code></pre>
  
  <p>when Jetty starts there are just 10 FIFOs, after a few days there are hundreds of them.</p>
  
  <p>I know it's a bit vague at this stage, but do you have any suggestions on where to look next, or how to get more detailed info about those file descriptors?</p>