I am working with a very large (~11GB) text file on a Linux system.  I am running it through a program which is checking the file for errors.  Once an error is found, I need to either fix the line or remove the line entirely.  And then repeat...
Eventually once I'm comfortable with the process, I'll automate it entirely.  For now however, let's assume I'm running this by hand.
What would be the fastest (in terms of execution time) way to remove a specific line from this large file?  I thought of doing it in Python...but would be open to other examples.  The line might be anywhere in the file.
If Python, assume the following interface:
def removeLine(filename, lineno):
Thanks,
-aj
If the lines are variable length then I don't believe that there is a better algorithm than reading the file line by line and writing out all lines, except for the one(s) that you do not want.
You can identify these lines by checking some criteria, or by keeping a running tally of lines read and suppressing the writing of the line(s) that you do not want.
If the lines are fixed length and you want to delete specific line numbers, then you may be able to use seek to move the file pointer... I doubt you're that lucky though.
Fastest Way to Delete a Line from Large File in Python