title: Jersey service file upload causes OutOfMemoryError
body: <p>I'm developing form submission service with Jersey 2.0. The form includes several <em>text</em> fields and one <em>file</em> field. I need to extract <em>file</em>, <em>file name</em>, <em>file media type</em> and <em>file content type</em> and save them in object store.</p>
  
  <pre><code>@Path("upload")
  @Consumes({MediaType.MULTIPART_FORM_DATA})
  @Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
  public class UploadService {
      @POST
      public BlobDo uploadFile(FormDataMultiPart uploadedBody) {
          String accountSid = uploadedBody.getField("account-sid").getValue();
          String apiToken = uploadedBody.getField("api-token").getValue();
          String checksum = uploadedBody.getField("checksum").getValue();
  
          FormDataBodyPart bodyPart = uploadedBody.getField("file");
          MySwiftObject obj = new MySwiftObject(bodyPart.getValueAs(InputStream.class));
          obj.setName(bodyPart.getContentDisposition().getFileName());
          obj.setContentType(bodyPart.getMediaType().toString());
          obj.setContentDisposition(bodyPart.getContentDisposition().toString());
     ...
  }
  </code></pre>
  
  <p>pom.xml</p>
  
  <pre><code>&lt;jersey.version&gt;2.17&lt;/jersey.version&gt;
  
  &lt;dependency&gt;
      &lt;groupId&gt;org.glassfish.jersey.containers&lt;/groupId&gt;
      &lt;artifactId&gt;jersey-container-servlet-core&lt;/artifactId&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
      &lt;groupId&gt;org.glassfish.jersey.media&lt;/groupId&gt;
      &lt;artifactId&gt;jersey-media-json-jackson&lt;/artifactId&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
      &lt;groupId&gt;org.glassfish.jersey.test-framework.providers&lt;/groupId&gt;
      &lt;artifactId&gt;jersey-test-framework-provider-inmemory&lt;/artifactId&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
      &lt;groupId&gt;org.glassfish.jersey.media&lt;/groupId&gt;
      &lt;artifactId&gt;jersey-media-multipart&lt;/artifactId&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
      &lt;groupId&gt;org.glassfish.jersey.core&lt;/groupId&gt;
      &lt;artifactId&gt;jersey-server&lt;/artifactId&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
      &lt;groupId&gt;org.glassfish.jersey.ext&lt;/groupId&gt;
      &lt;artifactId&gt;jersey-bean-validation&lt;/artifactId&gt;
  &lt;/dependency&gt;
  </code></pre>
  
  <p>form submission request</p>
  
  <pre><code>POST /nbs/v2/upload HTTP/1.1
  Host: 127.0.0.1:8080
  Cache-Control: no-cache
  Postman-Token: a4c1d4e9-5f71-2321-3870-e9cac0524f8d
  Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryA2Z9pPMA7y3da8BG
  
  ------WebKitFormBoundaryA2Z9pPMA7y3da8BG
  Content-Disposition: form-data; name="account-sid"
  
  Q45Ppm5ukvdqjTQ6eW0O5ztTXipwnjKQx1p6cf+fbCQ=
  ------WebKitFormBoundaryA2Z9pPMA7y3da8BG
  Content-Disposition: form-data; name="api-token"
  
  6397cd691909fdc14cef67dbc1dc2dc3
  ------WebKitFormBoundaryA2Z9pPMA7y3da8BG
  Content-Disposition: form-data; name="file"; filename="screen_4_100155.jpg"
  Content-Type: image/jpeg
  
  ......Exif..MM.*.............................b...........j
  ------WebKitFormBoundaryA2Z9pPMA7y3da8BG
  Content-Disposition: form-data; name="checksum"
  
  6a3381b1d16bded4a3dfc325a8bb800e
  ------WebKitFormBoundaryA2Z9pPMA7y3da8BG
  </code></pre>
  
  <p>JVM heap size</p>
  
  <pre><code>-Xmx=1024mb
  </code></pre>
  
  <hr>
  
  <h1>The problem</h1>
  
  <p>When uploading ~50MB file two temporary files with similar MD5 sums are created in the directory <code>/tmp/tomcat7-tomcat7-tmp</code> with name <code>FileBackedOutputStream7949386530699987086.tmp</code> and <code>MIME8234229766850016150.tmp</code></p>
  
  <p>Before upload is complete server throws exception</p>
  
  <pre>
  javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.OutOfMemoryError: Java heap space
      org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:421)
      org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:386)
      org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:335)
      org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:222)
  </pre>
  
  <p>and deletes the file with name <code>MIME8234229766850016150.tmp</code> but leaves the other one. And undeleted <code>FileBackedOutputStream....tmp</code> files fill the whole space on hard drive.</p>
  
  <hr>
  
  <h1>What I did</h1>
  
  <ol>
  <li><p>Increase heap space to 7GB, but ~200MB files can't be uploaded.</p></li>
  <li><p>Run job on server to delete old temporary files.</p></li>
  <li><p>Created file with name <code>jersey-multipart-config.properties</code> and content</p>
  
  <p><code>jersey.config.multipart.bufferThreshold = -1</code></p></li>
  </ol>
  
  <p>The file <code>MIME[random numbers].tmp</code> is no longer created, but <code>FileBackedOutputStream[random number].tmp</code> still hangs on hard drive unless tomcat is restarted.</p>
  
  <hr>
  
  <h1>Question</h1>
  
  <ol>
  <li><p>How can Jersey handle large files (maybe 1GB) without leaving temporary files on my hard disk? Best case would be not to use hard drive at all and transfer small chunks through memory.</p></li>
  <li><p>Why do I get <strong>heap overflow</strong> if input stream is backed with files?</p></li>
  </ol>
  
  <hr>
  
  <h1>Materials I read</h1>
  
  <ul>
  <li>The closes explanation I found so far. <a href="http://davidbuccola.blogspot.com/2009/09/configure-jersey-chunked-encoding-when.html?showComment=1435584267640#c8782205351685292800" rel="nofollow noreferrer">read</a></li>
  <li>This guy has similar problem but on the client side. <a href="https://stackoverflow.com/q/11176824/660408">read</a></li>
  <li>May contain solution, but couldn't understand the answer. <a href="https://stackoverflow.com/q/14864939/660408">read</a></li>
  <li>Pretty close to my problem but can't work it out. <a href="http://jersey.576304.n2.nabble.com/Jersey2-Client-OutOfMemoryError-when-uploading-large-file-td7581068.html" rel="nofollow noreferrer">read</a></li>
  <li><em>bufferThreshold</em> idea was taken from here. <a href="https://stackoverflow.com/a/6304378/660408">read</a></li>
  </ul>