title: Hibernate queries much slower with flushMode=AUTO until clear() is called
body: <p>I have a long running (but rather simple) application that uses Hibernate (via JPA).  It was experiencing rather <em>dramatic</em> slowdown as it ran.  I've been able to narrow down to requiring an occasional <code>entityManager.clear()</code> call.  When Hibernate's entity manager is tracking 100,000 entities, it is ~100x slower than when it is tracking only a few (see results below).  <strong>My question is: <em>why</em> does Hiberate slow down so much when it is tracking a lot of entities?  And are there any other ways around it?</strong></p>
  
  <hr>
  
  <p><strong>!!! Update: I've been able to narrow this down to Hibernate's auto-flushing code. !!!</strong> </p>
  
  <p>Specifically to <code>org.hibernate.event.internal.AbstractFlushingEventListener</code>'s <code>flushEntities()</code> method (at least in Hibernate 4.1.1.Final).  In it there is a loop that iterates over <strong>ALL</strong> entities in the persistence context, performing some extensive checks around flushing each of them (even though all the entities are already flushed in my example!).</p>
  
  <p>So partially answering the second part of my question, the performance problem can be addressed by setting the flush mode to <code>FlushModeType.COMMIT</code> on the query (see updated results below).  e.g.</p>
  
  <pre class="lang-java prettyprint-override"><code>Place place = em.createQuery("from Place where name = :name", Place.class)
      .setParameter("name", name)
      .setFlushMode(FlushModeType.COMMIT)  // &lt;-- yay!
      .getSingleResult();
  </code></pre>
  
  <p>... but this seems like a rather ugly solution--passing the responsibility for knowing if things are flushed to the query methods instead of keeping it in the updating methods.  It also pretty much means that I either have to set flush mode to COMMIT on all query methods, or, more likely, set it on the EntityManager.</p>
  
  <p>This makes me wonder: is this expected behavior?  Am I doing something wrong with flushing or how I define entities?  Or is this a limitation of (or possibly bug in) Hibernate? </p>
  
  <hr>
  
  <p>The sample code that I used to isolate the problem follows:</p>
  
  <h2>The Test Entity</h2>
  
  <pre class="lang-java prettyprint-override"><code>@Entity @Table(name="place") @Immutable
  public class Place {
      private Long _id;
      private String _name;
  
      @Id @GeneratedValue
      public Long getId() { return _id; }
      public void setId(Long id) { _id = id; }
  
      @Basic(optional=false) @Column(name="name", length=700,
          updatable=false, nullable=false, unique=true,
          columnDefinition="varchar(700) character set 'ascii' not null")
      public String getName() { return _name; }
      public void setName(String name) { _name = name; }
  
      @Override
      public boolean equals(Object o) { /* ... */ }
  
      @Override
      public int hashCode() { return getName().hashCode(); }
  }
  </code></pre>
  
  <h2>The Benchmark Code</h2>
  
  <p>The test code I have generates 100000 random place names and inserts them.  Then queries out 5000 of those randomly by name.  There is an index on the name column.</p>
  
  <pre class="lang-java prettyprint-override"><code>Place place = em.createQuery(
      "select p from Place p where p.name = :name", Place.class)
      .setParameter("name", name)
      .getSingleResult();
  </code></pre>
  
  <p>For comparison, and to make sure that it wasn't something in the database, I ran the following JDBC based query (under <code>em.unwrap(Session.class).doWork(...)</code>) over a separate randomly selected 5000 place names:</p>
  
  <pre class="lang-java prettyprint-override"><code>PreparedStatement ps = c.prepareStatement(
      "select id, name from place where name = ?");
  ps.setString(1, name);
  ResultSet rs = ps.executeQuery();
  while (rs.next()) {
      Place place = new Place();
      place.setId(rs.getLong(1));
      place.setName(rs.getString(2));
  }
  rs.close();
  ps.close();
  </code></pre>
  
  <p>(Note, I do create and close a PreparedStatement for each of the 5000 queries for the benchmark).</p>
  
  <h2>The Results</h2>
  
  <p>All results below are averaged over 5000 queries.  The JVM was given <code>-Xmx1G</code></p>
  
  <pre class="lang-none prettyprint-override"><code>Seconds/Query    Approach
  0.000160s        JDBC
  0.000286s        Hibernate calling clear() after import and every 100 queries
  0.000653s        Hibernate calling clear() once after the import
  0.012533s        Hibernate w/o calling clear() at all
  0.000292s        Hibernate w/o calling clear(), and with flush-mode COMMIT
  </code></pre>
  
  <p>Other observations: the during the Hibernate queries (w/o any clear calls), the java process pegged a core at near 100% utilization.  The JVM never exceeded 500MB heap.  There was a lot of GC activity during the queries too, but CPU utilization was clearly dominated by the Hibernate code.</p>