title: Best way to process database in chunks with Django QuerySet?
body: <p>I am running a batch operation over all rows in a database. This involves selecting every single model and doing something to it. It makes sense to split this into chunks and do it chunk by chunk.</p>
  
  <p>I'm currently using Paginator, because it's convenient. This means that I need an ordering on the values so that they can be paged through in order. This does generate SQL statements which  have <code>order</code> and <code>limit</code> clauses, and for each chunk I think Postgres may be sorting the whole table (although I can't claim to have any knowledge about internals). All I know is that the database is at around 50% CPU and I think that's too high just to be doing <code>select</code>s.</p>
  
  <p>What's the best way to iterate over the entire table in a RDMBS/CPU-friendly way?</p>
  
  <p>Assuming that the contents of the database doesn't change during the batch operation.</p>