title: doctrine2 - How to improve flush efficiency?
body: <p>I have to update my Doctrine entities to match records inside of (potentionaly very large) XML file. I have to also update ManyToMany associations according to data in the XML. This is what I do inside of a loop:</p>
  
  <ol>
  <li>get data from XML </li>
  <li>get entity from DB (if does not exist create new)</li>
  <li>set new entity properties</li>
  <li>get current entity associations (getter returns <code>ArrayCollection</code> object)</li>
  <li>clear all associations (by calling the <code>ArrayCollection::clear()</code>)</li>
  <li>set new associations (by calling <code>ArrayCollection::add()</code> in sub-loop)</li>
  <li>persist entity by EntityManager</li>
  </ol>
  
  <p><strong>After</strong> the loop I call <code>EntityManager::flush()</code>. </p>
  
  <p>The problem is that flushing generates large amount of queries instead of updating/inserting/deleting multiple rows at once. For every entity are executed following queries:</p>
  
  <ul>
  <li>SELECT to get entity from DB</li>
  <li>UPDATE to update entity properties (this is actually skipped now as no properties changed ... yet)</li>
  <li>DELETE to clear previous associations</li>
  <li>INSERT to insert new associations</li>
  </ul>
  
  <p>So in total for 305 records in XML i get 915 queries (I guess it could go up to 1220 queries if all entities would changed) which makes the import very slow.</p>
  
  <p>I could take advantage of IdentityMap and pre-fetch entities before loop, but there are still the UPDATE/DELETE/INSERT queries.</p>
  
  <ul>
  <li>Is there a way to let the flush method better optimize queries (use multi-insert, WHERE IN instead of multiple DELETE queries, etc.)?</li>
  <li>Is this normal behaviour of flush method or am I doing something wrong?</li>
  <li>Perhaps there is problem in the way how I update the associations of entity. Is there better way how to do this? (instead of "get/clear/add" method)</li>
  <li>I am aware of that Doctrine is not intended for mass betch processing, but I think using it for XML imports is the best way how to avoid DB inconsitencies which could appear with a not-ORM approach. Is that right?</li>
  <li>If the approach above is wrong, how should I solve the problem?</li>
  </ul>