title: Getting output from and giving commands to a python subprocess
body: <p>I am trying to get output from a subprocess and then give commands to that process based on the preceding output. I need to do this a variable number of times, when the program needs further input. (I also need to be able to hide the subprocess command prompt if possible). </p>
  
  <p>I figured this would be an easy task given that I have seen this problem being discussed in posts from 2003 and it is nearly 2012 and it appears to be a pretty common need and really seems like it should be a basic part of any programming language. Apparently I was wrong and somehow almost 9 years later there is still no standard way of accomplishing this task in a stable, non-destructive, platform independent way!</p>
  
  <p>I don't really understand much about file i/o and buffering or threading so I would prefer a solution that is as simple as possible. If there is a module that accomplishes this that is compatible with python 3.x, I would be very willing to download it. I realize that there are multiple questions that ask basically the same thing, but I have yet to find an answer that addresses the simple task that I am trying to accomplish.</p>
  
  <p>Here is the code I have so far based on a variety of sources; however I have absolutely no idea what to do next. All my attempts ended in failure and some managed to use 100% of my CPU (to do basically nothing) and would not quit.    </p>
  
  <pre><code>import subprocess
  from subprocess import Popen, PIPE
  p = Popen(r'C:\postgis_testing\shellcomm.bat',stdin=PIPE,stdout=PIPE,stderr=subprocess.STDOUT shell=True)
  stdout,stdin = p.communicate(b'command string')
  </code></pre>
  
  <p>In case my question is unclear I am posting the text of the sample batch file that I demonstrates a situation in which it is necessary to send multiple commands to the subprocess (if you type an incorrect command string the program loops).</p>
  
  <pre><code>@echo off
  :looper
  set INPUT=
  set /P INPUT=Type the correct command string:
  if "%INPUT%" == "command string" (echo you are correct) else (goto looper)
  </code></pre>
  
  <p>If anyone can help me I would very much appreciate it, and I'm sure many others would as well!</p>
  
  <p>EDIT here is the functional code using eryksun's code (next post) :</p>
  
  <pre><code>import subprocess
  import threading
  import time
  import sys
  
  try: 
      import queue
  except ImportError:
      import Queue as queue
  
  def read_stdout(stdout, q, p):
      it = iter(lambda: stdout.read(1), b'')
      for c in it:
          q.put(c)
          if stdout.closed:
              break
  
  _encoding = getattr(sys.stdout, 'encoding', 'latin-1')
  def get_stdout(q, encoding=_encoding):
      out = []
      while 1:
          try:
              out.append(q.get(timeout=0.2))
          except queue.Empty:
              break
      return b''.join(out).rstrip().decode(encoding)
  
  def printout(q):
      outdata = get_stdout(q)
      if outdata:
          print('Output: %s' % outdata)
  
  if __name__ == '__main__':
      #setup
      p = subprocess.Popen(['shellcomm.bat'], stdin=subprocess.PIPE, 
                       stdout=subprocess.PIPE, stderr=subprocess.PIPE, 
                       bufsize=0, shell=True) # I put shell=True to hide prompt
      q = queue.Queue()
      encoding = getattr(sys.stdin, 'encoding', 'utf-8')
  
      #for reading stdout
      t = threading.Thread(target=read_stdout, args=(p.stdout, q, p))
      t.daemon = True
      t.start()
  
      #command loop
      while p.poll() is None:
          printout(q)
          cmd = input('Input: ')
          cmd = (cmd + '\n').encode(encoding)
          p.stdin.write(cmd)
          time.sleep(0.1) # I added this to give some time to check for closure (otherwise it doesn't work)
  
      #tear down
      for n in range(4):
          rc = p.poll()
          if rc is not None:
              break
          time.sleep(0.25)
      else:
          p.terminate()
          rc = p.poll()
          if rc is None:
              rc = 1
  
      printout(q)
      print('Return Code: %d' % rc)
  </code></pre>
  
  <p>However when the script is run from a command prompt the following happens:</p>
  
  <pre><code>C:\Users\username&gt;python C:\postgis_testing\shellcomm7.py
  Input: sth
  Traceback (most recent call last):
  File "C:\postgis_testing\shellcomm7.py", line 51, in &lt;module&gt;
      p.stdin.write(cmd)
  IOError: [Errno 22] Invalid argument
  </code></pre>
  
  <p>It seems that the program closes out when run from command prompt. any ideas?</p>