Remove first 300M lines from 700 GB file on 1TB disk
My first idea was to solve this "queue" (FIFO) problem with a "two stacks" solution: Without special system calls, files can only be appended and truncated at the end, not the beginning, so they behave like a stack. Two stacks can be used to simulate a queue.<p>Implementation sketch:<p>1) Successively read the last 1 kiB from the source file, append it to a temporary file, and truncate 1 kiB from the source file.<p>2) Truncate the <i>last</i> 300M lines from the temporary file.<p>3) Successively read the last 1 kiB from the temporary file, append it to the destination file, and truncate 1 kiB from the temporary file.<p>This way, you never need more than 1 kiB of additional disk space (tune this number as desired). I think this is an <i>elegant</i> solution. The <i>correct</i> solution, of course, is to buy a second disk.