title: Fastest file reading in a multi-threaded application
body: <p>I have to read a 8192x8192 matrix into memory. I want to do it as fast as possible.<br>
  Right now I have this structure:</p>
  
  <pre><code>char inputFile[8192][8192*4]; // I know the numbers are at max 3 digits
  int8_t matrix[8192][8192]; // Matrix to be populated
  
  // Read entire file line by line using fgets
  while (fgets (inputFile[lineNum++], MAXCOLS, fp));
  
  //Populate the matrix in parallel, 
  for (t = 0; t &lt; NUM_THREADS; t++){
      pthread_create(&amp;threads[t], NULL, ParallelRead, (void *)t);
  }
  </code></pre>
  
  <p>In the function <code>ParallelRead</code>, I parse each line, do <code>atoi</code> and populate the matrix. The parallelism is line-wise like thread t parses line <code>t, t+ 1 * NUM_THREADS..</code></p>
  
  <p>On a two-core system with 2 threads, this takes</p>
  
  <pre><code>Loading big file (fgets) : 5.79126
  Preprocessing data (Parallel Read) : 4.44083
  </code></pre>
  
  <p>Is there a way to optimize this any further?</p>