Search and replace a line in a file in Python
<p>I want to loop over the contents of a text file and do a search and replace on some lines and write the result back to the file. I could first load the whole file in memory and then write it back, but that probably is not the best way to do it.</p>

<p>What is the best way to do this, within the following code?</p>

<pre><code>f = open(file)
for line in f:
    if line.contains('foo'):
        newline = line.replace('foo', 'bar')
        # how to write this newline back to the file
</code></pre>

<p>The shortest way would probably be to use the <a href="http://docs.python.org/2/library/fileinput.html?highlight=fileinput#fileinput">fileinput module</a>. For example, the following adds line numbers to a file, in-place:</p>

<pre><code>import fileinput

for line in fileinput.input("test.txt", inplace=True):
    print "%d: %s" % (fileinput.filelineno(), line),
</code></pre>

<p>What happens here is:</p>

<ol>
<li>The original file is moved to a backup file</li>
<li>The standard output is redirected to the original file within the loop</li>
<li>Thus any <code>print</code> statements write back into the original file</li>
</ol>

<p><code>fileinput</code> has more bells and whistles. For example, it can be used to automatically operate on all files in <code>sys.args[1:]</code>, without your having to iterate over them explicitly. Starting with Python 3.2 it also provides a convenient context manager for use in a <code>with</code> statement.</p>

<hr>

<p>While <code>fileinput</code> is great for throwaway scripts, I would be wary of using it in real code because admittedly it's not very readable or familiar. In real (production) code it's worthwhile to spend just a few more lines of code to make the process explicit and thus make the code readable.</p>

<p>There are two options:</p>

<ol>
<li>The file is not overly large, and you can just read it wholly to memory. Then close the file, reopen it in writing mode and write the modified contents back.</li>
<li>The file is too large to be stored in memory; you can move it over to a temporary file and open that, reading it line by line, writing back into the original file. Note that this requires twice the storage.</li>
</ol>
