title: Mimicing glib.spawn_async with Popen…
body: <p>The function <a href="http://www.pygtk.org/docs/pygobject/glib-functions.html#function-glib--spawn-async" rel="nofollow">glib.spawn_async</a> allows you to hook three callbacks which are called on event on <code>stdout</code>, <code>stderr</code>, and on process completion.</p>
  
  <p><em>How can I mimic the same functionality with <a href="https://docs.python.org/2/library/subprocess.html" rel="nofollow">subprocess</a> with either threads or asyncio?</em></p>
  
  <p>I am more interested in the functionality rather than threading/asynio but an answer that contains both will earn a bounty.</p>
  
  <p>Here is a toy program that shows what I want to do:</p>
  
  <pre><code>import glib
  import logging
  import os
  import gtk
  
  
  class MySpawn(object):
      def __init__(self):
          self._logger = logging.getLogger(self.__class__.__name__)
  
      def execute(self, cmd, on_done, on_stdout, on_stderr):
          self.pid, self.idin, self.idout, self.iderr = \
              glib.spawn_async(cmd,
                               flags=glib.SPAWN_DO_NOT_REAP_CHILD,
                               standard_output=True,
                               standard_error=True)
          fout = os.fdopen(self.idout, "r")
          ferr = os.fdopen(self.iderr, "r")
          glib.child_watch_add(self.pid, on_done)
          glib.io_add_watch(fout, glib.IO_IN, on_stdout)
          glib.io_add_watch(ferr, glib.IO_IN, on_stderr)
          return self.pid
  
  
  if __name__ == '__main__':
      logging.basicConfig(format='%(thread)d %(levelname)s:  %(message)s',
                          level=logging.DEBUG)
      cmd = '/usr/bin/git ls-remote https://github.com/DiffSK/configobj'.split()
  
      def on_done(pid, retval, *args):
          logging.info("That's all folks!…")
  
      def on_stdout(fobj, cond):
          """This blocks which is fine for this toy example…"""
          for line in fobj.readlines():
              logging.info(line.strip())
          return True
  
      def on_stderr(fobj, cond):
          """This blocks which is fine for this toy example…"""
          for line in fobj.readlines():
              logging.error(line.strip())
          return True
  
      runner = MySpawn()
      runner.execute(cmd, on_done, on_stdout, on_stderr)
      try:
          gtk.main()
      except KeyboardInterrupt:
          print('')
  </code></pre>
  
  <p>I should add that since <code>readlines()</code> is blocking, the above will buffer all the output and send it at once. If this is not what one wants, then you have to use <code>readline()</code> and make sure that on end of command you finish reading all the lines you did not read before.</p>