title: Asyncronous Bidirectional IO Redirection for a child process
body: <p>I am trying to figure out a generalized way for Asynchronous Bidirectional IO Redirection of a child process. Basically, I would like to spawn an interactive child process that waits for input and any output should be read back. I tried to experiment with python.subprocess by spawning a new python process. A base simplistic example tried to achieve is as follows</p>
  
  <pre><code>process = subprocess.Popen(['/usr/bin/python'],shell=False,stdin=subprocess.PIPE, stdout=subprocess.PIPE)
  while True:
      output = process.stdout.readline()
      print output
      input = sys.stdin.readline()
      process.stdin.write(input)
  </code></pre>
  
  <p>and executing the above code snippet simply hangs without any output. I tried running with <code>/usr/bash</code> and <code>/usr/bin/irb</code> but the result is all the same. My guess is, buffered IO is simply not gelling well with IO redirection.</p>
  
  <p>So my question is, is it feasible to read the output of a child process without flushing the buffer or quitting the subprocess?</p>
  
  <p>The <a href="https://stackoverflow.com/questions/1682911/reliable-bidirectional-communication-to-a-linux-process">following post</a> mentions IPC sockets but for that I would have to change the child process which may not be feasible. Is there any other way to achieve it?</p>
  
  <p>Note*** My ultimate goal is to create a server REPL process which can interact with a remote web client. Though the example given is of Python, my ultimate goal is to wrap all available REPL by a generalized wrapper. </p>
  
  <hr>
  
  <p>With the help of some of the suggestion in the answers I came up with the following</p>
  
  <pre><code>#!/usr/bin/python
  import subprocess, os, select
  proc = subprocess.Popen(['/usr/bin/python'],shell=False,stdin=subprocess.PIPE, stdout=subprocess.PIPE,stderr=subprocess.PIPE)
  for i in xrange(0,5):
      inputready, outputready, exceptready = select.select([proc.stdout, proc.stderr],[proc.stdout, proc.stderr],[proc.stdout, proc.stderr],0)
      if not inputready: print "No Data",
      print inputready, outputready, exceptready
      for s in inputready: print s.fileno(),s.readline()
  proc.terminate()
  print "After Terminating"
  for i in xrange(0,5):
      inputready, outputready, exceptready = select.select([proc.stdout, proc.stderr],[proc.stdout, proc.stderr],[proc.stdout, proc.stderr],0)
      if not inputready: print "No Data",
      print inputready, outputready, exceptready
      for s in inputready: print s.fileno(),s.readline() 
  </code></pre>
  
  <p>now, though the programs is not in deadlock but unfortunately there is no output. Running the above code I get</p>
  
  <pre><code>No Data [] [] []
  No Data [] [] []
  No Data [] [] []
  No Data [] [] []
  No Data [] [] []
  After Terminating
  No Data [] [] []
  No Data [] [] []
  No Data [] [] []
  No Data [] [] []
  No Data [] [] []
  </code></pre>
  
  <p>Just FYI,
  running python as </p>
  
  <pre><code>/usr/bin/python 2&gt;&amp;1|tee test.out
  </code></pre>
  
  <p>seems to be working just fine.</p>
  
  <p>I also came up with a 'C' code. But the result is not different.</p>
  
  <pre><code>int kbhit() {
      struct timeval tv;
      fd_set fds;
      tv.tv_sec = tv.tv_usec = 0;
      FD_ZERO(&amp;fds);
      FD_SET(STDIN_FILENO, &amp;fds);
      select(STDIN_FILENO+1, &amp;fds, NULL, NULL, &amp;tv);
      return FD_ISSET(STDIN_FILENO, &amp;fds);
  }
  void receive(char *str) {
      char ch;
      fprintf(stderr,"IN1\n");
      if(!kbhit()) return;
      fprintf(stderr,"IN2\n");
      fprintf(stderr,"%d\n",kbhit());
      for(;kbhit() &amp;&amp; (ch=fgetc(stdin))!=EOF;) {
          fprintf(stderr,"%c,%d",ch,kbhit());
      }
      fprintf(stderr,"Done\n");
  }
  int main(){
      pid_t pid;
      int rv, pipeP2C[2],pipeC2P[2];  
      pipe(pipeP2C);
      pipe(pipeC2P);
      pid=fork();
      if(pid){
          dup2(pipeP2C[1],1); /* Replace stdout with out side of the pipe */
          close(pipeP2C[0]);  /* Close unused side of pipe (in side) */
          dup2(pipeC2P[0],0); /* Replace stdin with in side of the pipe */
          close(pipeC2P[1]);  /* Close unused side of pipe (out side) */
          setvbuf(stdout,(char*)NULL,_IONBF,0);   /* Set non-buffered output on stdout */
          sleep(2);
          receive("quit()\n");
          wait(&amp;rv);              /* Wait for child process to end */
          fprintf(stderr,"Child exited with a %d value\n",rv);
      }
      else{
          dup2(pipeP2C[0],0); /* Replace stdin with the in side of the pipe */
          close(pipeP2C[1]);  /* Close unused side of pipe (out side) */
          dup2(pipeC2P[1],1); /* Replace stdout with the out side of the pipe */
          close(pipeC2P[0]);  /* Close unused side of pipe (out side) */
          setvbuf(stdout,(char*)NULL,_IONBF,0);   /* Set non-buffered output on stdout */
          close(2), dup2(1,2); /*Redirect stderr to stdout */
          if(execl("/usr/bin/python","/usr/bin/python",NULL) == -1){
              fprintf(stderr,"execl Error!");
              exit(1);
          }
      }
      return 0;
  }
  </code></pre>