title: Can I somehow share an asynchronous queue with a subprocess?
body: <p>I would like to use a queue for passing data from a parent to a child process which is launched via <code>multiprocessing.Process</code>. However, since the parent process uses Python's new <code>asyncio</code> library, the queue methods need to be non-blocking. As far as I understand, <code>asyncio.Queue</code> is made for inter-task communication and cannot be used for inter-process communication. Also, I know that <code>multiprocessing.Queue</code> has the <code>put_nowait()</code> and <code>get_nowait()</code> methods but I actually need coroutines that would still block the current task (but not the whole process). Is there some way to create coroutines that wrap <code>put_nowait()</code>/<code>get_nowait()</code>? On another note, are the threads that <code>multiprocessing.Queue</code> uses internally compatible after all with an event loop running in the same process?</p>
  
  <p>If not, what other options do I have? I know I could implement such a queue myself by making use of asynchronous sockets but I hoped I could avoid that… </p>
  
  <p><strong>EDIT:</strong>
  I also considered using pipes instead of sockets but it seems <code>asyncio</code> is not compatible with <a href="https://docs.python.org/dev/library/multiprocessing.html#multiprocessing.Pipe" rel="noreferrer"><code>multiprocessing.Pipe()</code></a>. More precisely, <code>Pipe()</code> returns a tuple of <a href="https://docs.python.org/dev/library/multiprocessing.html#multiprocessing.Connection" rel="noreferrer"><code>Connection</code></a> objects which are <em>not</em> file-like objects. However, <code>asyncio.BaseEventLoop</code>'s methods <a href="https://docs.python.org/3.4/library/asyncio-eventloop.html#watch-file-descriptors" rel="noreferrer"><code>add_reader()</code>/<code>add_writer()</code></a> methods and <a href="https://docs.python.org/3.4/library/asyncio-eventloop.html#connect-pipes" rel="noreferrer"><code>connect_read_pipe()</code>/<code>connect_write_pipe()</code></a> all expect file-like objects, so it is impossible to asynchronously read from/write to such a <code>Connection</code>. In contrast, the usual file-like objects that the <code>subprocess</code> package uses as pipes pose no problem at all and <a href="https://code.google.com/p/tulip/source/browse/examples/child_process.py" rel="noreferrer">can easily be used in combination with <code>asyncio</code></a>.</p>
  
  <p><strong>UPDATE:</strong>
  I decided to explore the pipe approach a bit further: I converted the <code>Connection</code> objects  returned by <code>multiprocessing.Pipe()</code> into file-like objects by retrieving the file descriptor via <a href="https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Connection.fileno" rel="noreferrer"><code>fileno()</code></a> and passing it to <code>os.fdopen()</code>. Finally, I passed the resulting file-like object to the event loop's <code>connect_read_pipe()</code>/<code>connect_write_pipe()</code>. (There is some <a href="https://groups.google.com/forum/#!topic/python-tulip/6tRswyRhwE4" rel="noreferrer">mailing list discussion</a> on a related issue if someone is interested in the exact code.) However, <code>read()</code>ing the stream gave me an <code>OSError: [Errno 9] Bad file descriptor</code> and I didn't manage to fix this. Also considering the <a href="https://docs.python.org/3/library/asyncio-eventloops.html#windows" rel="noreferrer">missing support for Windows</a>, I will not pursue this any further. </p>