title: Working with large arrays - OutOfRam
body: <p>I have an algorithm where I create two bi-dimensional arrays like this:</p>
  
  <pre><code>  TYPE
     TPtrMatrixLine = array of byte;
     TCurMatrixLine = array of integer;
     TPtrMatrix     = array of TPtrMatrixLine;                                    
     TCurMatrix     = array of TCurMatrixLine;  
  
  
    function x
    var  
       PtrsMX: TPtrMatrix;
       CurMx : TCurMatrix;         
    begin 
     { Try to allocate RAM }
     SetLength(PtrsMX, RowNr+1, ColNr+1);                              
     SetLength(CurMx , RowNr+1, ColNr+1);
     for all rows do 
      for all cols do 
       FillMatrixWithData; &lt;------- CPU intensive task. It could take up to 10-20 min
    end;
  </code></pre>
  
  <p>The two matrices have always the same dimension.
  Usually there are only 2000 lines and 2000 columns in the matrix but sometimes it can go as high as 25000x6000 so for both matrices I need something like 146.5 + 586.2 = 732.8MB of RAM.
  The problem is that the two blocks need to be contiguous so in most cases, even if 500-600MB of free RAM doesn't seem much on a modern computer, I run out of RAM.</p>
  
  <p>The algorithm fills the cells of the array with data based on the neighbors of that cell. The operations are just additions and subtractions.</p>
  
  <p>The TCurMatrixLine is the one that takes a lot or RAM since it uses integers to store data. Unfortunately, values stored may have sign so I cannot use Word instead of integers. SmallInt is too small (my values are bigger than SmallInt, but smaller than Word). I hope that if there is any other way to implement this, it needs not to add a lot of overhead, since processing a matrix with so many lines/column already takes a lot of time. In other words I hope that decreasing memory requirements will not increase processing time.</p>
  
  <p>Any idea how to decrease the memory requirements?
  [I use Delphi 7]</p>
  
  <hr>
  
  <p><strong>Update</strong>
  Somebody suggested that each row of my array should be an independent uni-dimensional array.
  I create as many rows (arrays) as I need and store them in TList. Sound very good. Obviously there will be no problem allocation such small memory blocks. But I am afraid it will have a gigantic impact on speed. I use now </p>
  
  <pre><code>TCurMatrixLine = array of integer;                                   
  TCurMatrix     = array of TCurMatrixLine; 
  </code></pre>
  
  <p>because it is faster than <code>TCurMatrix= array of array of integer</code> (because of the way data is placed in memory). So, breaking the array in independent lines may affect the speed.</p>