title: Intercepting subprocess.Popen call in Python
body: <p>I'm writing a functional test for a legacy Python script so that I can make a one-line change to it without being paralysed by fear. ;)</p>
  
  <p>The script in question invokes wget(1) using <em>subprocess.Popen</em> to download an XML file which is then parsed:</p>
  
  <pre><code>def download_files():
      os.mkdir(FEED_DIR)
      os.chdir(FEED_DIR)
  
      wget_process = Popen(
          ["wget", "--quiet", "--output-document", "-", "ftp://foo.com/bar.tar"],
          stdout=PIPE
      )
      tar_process = Popen(["tar", "xf", "-"], stdin=wget_process.stdout)
      stdout, stderr = tar_process.communicate()
  </code></pre>
  
  <p>Obviously, it would be preferable to modify the script to use an HTTP library instead of exec-ing wget, but as I said, it is a legacy script, so I need to keep my change minimal and absolutely focused on the business requirement, which has nothing to do with how the XML file is obtained.</p>
  
  <p>The obvious solution to me is to intercept the call to <em>subprocess.Popen</em> and return my own test XML. <a href="https://stackoverflow.com/questions/2704434/intercept-method-calls-in-python" title="foo">Intercept method calls in Python</a> demonstrates how to use <em>setattr</em> to do this, but I must be missing something:</p>
  
  <pre><code>Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) 
  [GCC 4.4.5] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  &gt;&gt;&gt; import subprocess
  &gt;&gt;&gt; object.__getattribute__(subprocess, 'Popen')
  &lt;class 'subprocess.Popen'&gt;
  &gt;&gt;&gt; attr = object.__getattribute__(subprocess, 'Popen')
  &gt;&gt;&gt; hasattr(attr, '__call__')
  True
  &gt;&gt;&gt; def foo(): print('foo')
  ... 
  &gt;&gt;&gt; foo
  &lt;function foo at 0x7f8e3ced3c08&gt;
  &gt;&gt;&gt; foo()
  foo
  &gt;&gt;&gt; setattr(subprocess, '__call__', foo)
  &gt;&gt;&gt; getattr(subprocess, '__call__')
  &lt;function foo at 0x7f8e3ced3c08&gt;
  &gt;&gt;&gt; subprocess.Popen([ r"tail", "-n 1", "x.txt" ], stdout = subprocess.PIPE)
  &lt;subprocess.Popen object at 0x7f8e3ced9cd0&gt;
  &gt;&gt;&gt; tail: cannot open `x.txt' for reading: No such file or directory
  </code></pre>
  
  <p>As you can see, the real <em>subprocess.Popen</em> is being called, despite the attribute being set correctly (at least to my largely untrained eye). Is this just a result of running this in interactive Python, or should I expect the same result from dropping this sort of code into my test script:</p>
  
  <pre><code>class MockProcess:
    def __init__(self, output):
      self.output = output
  
    def stderr(): pass
    def stdout(): return self.output
  
    def communicate():
      return stdout, stderr
  
  
  # Runs script, returning output
  #
  def run_agent():
    real_popen = getattr(subprocess.Popen, '__call__')
    try:
      setattr(subprocess.Popen, '__call__', lambda *ignored: MockProcess('&lt;foo bar="baz" /&gt;')
      )
      return real_popen(['myscript.py'], stdout = subprocess.PIPE).communicate()[0]
    finally:
      setattr(subprocess.Popen, '__call__', real_popen)
  </code></pre>