title: Processing huge text files
body: <p><strong>Problem:</strong>
  I've a huge raw text file (assume of 3gig), I need to go through each word in the file
  and find out that a word appears how many times in the file.</p>
  
  <p><strong>My Proposed Solution:</strong>
  Split the huge file into multiple files and each splitted file will have words in a sorted manner. For example,
  all the words starting with "<em>a</em>" will be stored in a "<em>_a.dic</em>" file. So, at any time we will not execeed more than 26 files.</p>
  
  <p>The problem in this approach is,</p>
  
  <p>I can use streams to read the file, but wanted to use threads to read certain parts of the file. For example, read 0-1024 bytes with a separate thread (atleast have 4-8 threads based on the no. of processors exist in the box). Is this is possible or am I dreaming?</p>
  
  <p>Any better approach?</p>
  
  <p>Note: It should be a pure c++ or c based solution. No databases etc., are allowed.</p>