title: Python: Respond to Command Line Prompts
body: <p>I am trying to use Python to interact with another program via the command line.  The main problem I am having is a specific call that has multiple follow-up prompts.  Initially the command line call asks for the name of a project and then proceeds to ask if I would like to view any of the subfolders of the project.  I need to answer y/n to each of these in order and the answer to each is unfortunately not all y or n.  Additionally, I cannot know the answer to the question without reading the individual prompts so I am incapable of sending a block of 'y's or 'n's all at once.</p>
  
  <p>This is the command line call: </p>
  
  <p><strong>si viewproject</strong></p>
  
  <p>After entering the command, the command line prompts: </p>
  
  <p><strong>Enter the project name:</strong></p>
  
  <p>And an example response would be: </p>
  
  <p><strong>Enter the project name: c:/test.pj</strong></p>
  
  <p>After entering the project, it prompts the following: </p>
  
  <p><strong>Do you want to recurse into the subproject test_subprj.pj? [ynYN](n)</strong></p>
  
  <p>At which point I need to respond with either a y or n depending on if I need that subproject.  Again, the response to this question is dependent on the subproject. I need to be able to read the subproject in this prompt in order to respond to it with a 'y' or 'n'</p>
  
  <p>Currently I need to manually enter in the project and each of the y's and n's respectively.  My goal is to automate this process using Python.</p>
  
  <p>Is there a way to respond to these command line prompts automatically?</p>
  
  <h1>Current Progress</h1>
  
  <h2>Subprocess Strategy</h2>
  
  <pre><code> project_path = "c:/test.pj"
  
   with Popen(["si", "viewproject", "--project=" + project_path], 
               stdin=PIPE, stdout=PIPE, universal_newlines=True) as p:
       for line in p.stdout: 
           if line.startswith("Do you want"):
               answer = 'n'
           else:
               continue # skip it
           print(answer, file=p.stdin) # provide answer
           p.stdin.flush()
  </code></pre>
  
  <p>This method is hanging after the with Popen statement.  It never errors, but it never enters or exits the for statement and never completes.  Currently I am defaulting all answers to "n", but that will be replaced with logic later.</p>
  
  <h2>Winpexpect Strategy</h2>
  
  <pre><code> import re
   import sys
   from functools import partial
   import winpexpect
  
   project_path = "c:/test.pj"
  
   p = winpexpect.winspawn('si viewproject --project=' + project_path)
   p.logfile = sys.stdout
   patterns = [re.compile('ynYN'), winpexpect.EOF]
  
   for found in iter(partial(p.expect, patterns), 1): # until EOF
       if found == 0:
           answer = 'n'
           p.sendline(answer)
  </code></pre>
  
  <p>Returns the following error message:</p>
  
  <pre><code> Traceback (most recent call last):
     File "C:\Python33\lib\site-packages\winpexpect-1.5-py3.3.egg\winpexpect.py", line 541, in read_nonblocking
       handle, status, data = self.child_output.get(timeout=timeout)
     File "C:\Python33\lib\queue.py", line 175, in get
       raise Empty
   queue.Empty
  
   During handling of the above exception, another exception occurred:
  
   Traceback (most recent call last):
     File "C:\Python33\lib\site-packages\winpexpect-1.5-py3.3.egg\pexpect.py", line 1378, in expect_loop
       c = self.read_nonblocking (self.maxread, timeout)
     File "C:\Python33\lib\site-packages\winpexpect-1.5-py3.3.egg\winpexpect.py", line 543, in read_nonblocking
       raise TIMEOUT('Timeout exceeded in read_nonblocking().')
   pexpect.TIMEOUT: Timeout exceeded in read_nonblocking().
  
   During handling of the above exception, another exception occurred:
  
   Traceback (most recent call last):
     File "K:\eclipse_3.6.0\plugins\org.python.pydev_2.6.0.2012062818\pysrc\pydev_runfiles.py", line 432, in __get_module_from_str
       mod = __import__(modname)
     File "C:\workspace\Test_prj\Test_prj.py", line 19, in &lt;module&gt;
       for found in iter(partial(p.expect, patterns), 1): # until EOF
     File "C:\Python33\lib\site-packages\winpexpect-1.5-py3.3.egg\pexpect.py", line 1311, in expect
       return self.expect_list(compiled_pattern_list, timeout, searchwindowsize)
     File "C:\Python33\lib\site-packages\winpexpect-1.5-py3.3.egg\pexpect.py", line 1325, in expect_list
       return self.expect_loop(searcher_re(pattern_list), timeout, searchwindowsize)
     File "C:\Python33\lib\site-packages\winpexpect-1.5-py3.3.egg\pexpect.py", line 1409, in expect_loop
       raise TIMEOUT (str(e) + '\n' + str(self))
   pexpect.TIMEOUT: Timeout exceeded in read_nonblocking().
   &lt;winpexpect.winspawn object at 0x0144AE50&gt;
   version: 2.3 ($Revision: 399 $)
   command: si
   args: ['si', 'viewproject', '--project=c:/test.pj']
   searcher: searcher_re:
       0: re.compile("ynYN")
       1: EOF
   buffer (last 100 chars): 
   before (last 100 chars): 
   after: &lt;class 'pexpect.TIMEOUT'&gt;
   match: None
   match_index: None
   exitstatus: None
   flag_eof: False
   pid: 6448
   child_fd: 4
   closed: False
   timeout: 30
   delimiter: &lt;class 'pexpect.EOF'&gt;
   logfile: &lt;_io.TextIOWrapper name='&lt;stdout&gt;' mode='w' encoding='Cp1252'&gt;
   logfile_read: None
   logfile_send: None
   maxread: 2000
   ignorecase: False
   searchwindowsize: None
   delaybeforesend: 0.05
   delayafterclose: 0.1
   delayafterterminate: 0.1
   ERROR: Module: Test_prj could not be imported (file: C:\workspace\Test_prj\Test_prj.py).
  </code></pre>
  
  <h2>Installing Winpexpect</h2>
  
  <p><strong>Lazy Persons Way</strong></p>
  
  <p><a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/#distribute" rel="noreferrer">Install Distribute</a></p>
  
  <p><a href="http://www.varunpant.com/posts/how-to-setup-easy_install-on-windows" rel="noreferrer">Do This</a></p>
  
  <p><a href="http://sourceforge.net/projects/pywin32/files/pywin32/Build%20218/" rel="noreferrer">Install PyWin32</a></p>
  
  <p><a href="https://bitbucket.org/geertj/winpexpect/wiki/Home" rel="noreferrer">Install Winpexpect</a></p>
  
  <p><a href="https://nose.readthedocs.org/en/latest/" rel="noreferrer">Optional: Install Nose</a></p>
  
  <p><a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/#pip" rel="noreferrer">Optional: Install Pip</a></p>
  
  <p><strong>First-World Problems</strong></p>
  
  <p>Python is a new language for me, and I had never installed a package before for Python. Additionally, Python 3.x is a little different than the other versions of Python making installing modules a little bit more of an adventure.</p>
  
  <p>So, to help others get some sweet sweet module action (and to help those who are more knowledgeable see if I did anything wrong) here's a soon to be success story (hopefully) documenting how I got and installed my first module.</p>
  
  <p><strong>Setup</strong></p>
  
  <p>Python allows third-party groups to develop and distribute modules that extend the abilities of the programming language. Naturally, there is a standard way to help third-party developers make modules as easily available to the end-user as possible.</p>
  
  <p>For Python 3.x, that standard for distributing modules is called Distutils.</p>
  
  <p>Here is how a developer uses Distutils:
  <a href="http://docs.python.org/3.2/distutils/index.html#distutils-index" rel="noreferrer">Distributing Python Modules</a></p>
  
  <p>And here is how the end-user uses Distutils:
  <a href="http://docs.python.org/3.2/install/" rel="noreferrer">Installing Python Modules</a></p>
  
  <p>Normally, navigating to the folder of your downloaded module in the command line and running "setup.py install" will be enough.</p>
  
  <p><strong>BUT</strong></p>
  
  <p>Sometimes life isn't so easy and you may still have problems with your installation. You may, in fact, need something else. For example, you may get the following error:</p>
  
  <p>"ImportError “No Module named Setuptools”"</p>
  
  <p>Luckily, there is a solution for that:
  <a href="https://stackoverflow.com/questions/14426491/python-3-importerror-no-module-named-setuptools">Python 3: ImportError "No Module named Setuptools"</a></p>
  
  <p>As it turns out, not everything uses distutils. Some packages use setuptools. Unfortunately, there is no setuptools for Python 3.x. Rather, Python 3.x uses distribute which is a branch of setuptools.</p>
  
  <p>So for those who use Python 3.x, here is Distribute: <a href="https://pypi.python.org/pypi/distribute" rel="noreferrer">Distribute</a></p>
  
  <p>For those using Python 2.x, here is Setuptools: <a href="https://pypi.python.org/pypi/setuptools" rel="noreferrer">Setuptools</a></p>
  
  <p>In the Installation Instructions for Distribute, it says the following:
  "Download
  <code>distribute_setup.py &lt;http://python-distribute.org/distribute_setup.py&gt;</code>_
  and execute it, using the Python interpreter of your choice."</p>
  
  <p>It also says: "Notice this file is also provided in the source release."</p>
  
  <p>So I downloaded Distribute and saved it to the computer.  Once it was saved to the computer, I ran distribute_setup.py from the source release and got the following error:</p>
  
  <pre><code>Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.36.tar.gz
  Traceback (most recent call last):
    File "C:\Python33\lib\urllib\request.py", line 1252, in do_open
      h.request(req.get_method(), req.selector, req.data, headers)       File "C:\Python33\lib\http\client.py", line 1049, in request
      self._send_request(method, url, body, headers)
    File "C:\Python33\lib\http\client.py", line 1087, in _send_request
      self.endheaders(body)
    File "C:\Python33\lib\http\client.py", line 1045, in endheaders
      self._send_output(message_body)
    File "C:\Python33\lib\http\client.py", line 890, in _send_output
      self.send(msg)
    File "C:\Python33\lib\http\client.py", line 828, in send
      self.connect()
    File "C:\Python33\lib\http\client.py", line 806, in connect
      self.timeout, self.source_address)
    File "C:\Python33\lib\socket.py", line 406, in create_connection
      for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  socket.gaierror: [Errno 11001] getaddrinfo failed
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "C:\workspace\PythonTest\distribute_setup.py", line 553, in &lt;module&gt;
      sys.exit(main())
    File "C:\workspace\PythonTest\distribute_setup.py", line 549, in main
      tarball = download_setuptools(download_base=options.download_base)
    File "C:\workspace\PythonTest\distribute_setup.py", line 204, in download_setuptools
      src = urlopen(url)
    File "C:\Python33\lib\urllib\request.py", line 160, in urlopen
      return opener.open(url, data, timeout)
    File "C:\Python33\lib\urllib\request.py", line 473, in open
      response = self._open(req, data)
    File "C:\Python33\lib\urllib\request.py", line 491, in _open
      '_open', req)
    File "C:\Python33\lib\urllib\request.py", line 451, in _call_chain
      result = func(*args)
    File "C:\Python33\lib\urllib\request.py", line 1272, in http_open
      return self.do_open(http.client.HTTPConnection, req)
    File "C:\Python33\lib\urllib\request.py", line 1255, in do_open
      raise URLError(err)
  urllib.error.URLError: &lt;urlopen error [Errno 11001] getaddrinfo failed&gt;
  </code></pre>
  
  <p>Well that is no good! I honestly still do not know where that error is coming from or why it happened.</p>
  
  <p>Regardless, then I found the following site that ran a .exe to install distribute as well as pip.</p>
  
  <p><a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/#distribute" rel="noreferrer">Install Distribute</a></p>
  
  <p><a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/#pip" rel="noreferrer">Install Pip</a></p>
  
  <p>So I got those installed and then used the following site to setup my computer to more easily use easy_install: <a href="http://www.varunpant.com/posts/how-to-setup-easy_install-on-windows" rel="noreferrer">Setting Up Easy Install Made Easy</a></p>
  
  <p>Once I got this working I then installed nose:<a href="https://nose.readthedocs.org/en/latest/" rel="noreferrer">Nose</a></p>
  
  <p>The reason I got nose was because the <a href="https://bitbucket.org/geertj/winpexpect/wiki/Home" rel="noreferrer">Winpexpect website</a> says:
  "WinPexpect includes unit tests. To run the tests, you need nose. Use the following command to run the tests:</p>
  
  <p>$ python setup.py test"</p>
  
  <p>Well that sounds nice :).  Now I just wished I knew where to run that test.  I know that if you install manually you use the setup.py install command so there will most definitely be a setup.py in the zipped directory online. To see if this was correct, I downloaded and saved the winpexpect file, extracted the information, navigated to it via command-line, and ran setup.py test.</p>
  
  <p>Here was the following result:</p>
  
  <pre><code>running test
  running build_py
  running egg_info
  creating c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info
  writing c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info\PKG-INFO
  writing dependency_links to c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info\dependency_links.txt
  writing top-level names to c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info\top_level.txt
  writing requirements to c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info\requires.txt
  writing manifest file 'c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info\SOURCES.txt'
  reading manifest file 'c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info\SOURCES.txt'
  writing manifest file 'c:\documents and settings\slz1fh\desktop\winpexpect\geertj-winpexpect-76df3cfcb143\build\lib\winpexpect.egg-info\SOURCES.txt'
  running build_ext
  Traceback (most recent call last):
    File "C:\Documents and Settings\SLZ1FH\Desktop\winpexpect\geertj-winpexpect-76df3cfcb143\setup.py", line 35, in &lt;module&gt;
      use_2to3 = True
    File "C:\Python33\lib\distutils\core.py", line 148, in setup
      dist.run_commands()
    File "C:\Python33\lib\distutils\dist.py", line 917, in run_commands
      self.run_command(cmd)
    File "C:\Python33\lib\distutils\dist.py", line 936, in run_command
      cmd_obj.run()
    File "C:\Python33\lib\site-packages\distribute-0.6.36-py3.3.egg\setuptools\command\test.py", line 138, in run
      self.with_project_on_sys_path(self.run_tests)
    File "C:\Python33\lib\site-packages\distribute-0.6.36-py3.3.egg\setuptools\command\test.py", line 118, in with_project_on_sys_path
      func()
    File "C:\Python33\lib\site-packages\distribute-0.6.36-py3.3.egg\setuptools\command\test.py", line 164, in run_tests
      testLoader = cks
    File "C:\Python33\lib\unittest\main.py", line 124, in __init__
      self.parseArgs(argv)
    File "C:\Python33\lib\unittest\main.py", line 168, in parseArgs
      self.createTests()
    File "C:\Python33\lib\unittest\main.py", line 175, in createTests
      self.module)
    File "C:\Python33\lib\unittest\loader.py", line 137, in loadTestsFromNames
      suites = [self.loadTestsFromName(name, module) for name in names]
    File "C:\Python33\lib\unittest\loader.py", line 137, in &lt;listcomp&gt;
      suites = [self.loadTestsFromName(name, module) for name in names]
    File "C:\Python33\lib\unittest\loader.py", line 96, in loadTestsFromName
      module = __import__('.'.join(parts_copy))
    File "C:\Python33\lib\site-packages\nose-1.3.0-py3.3.egg\nose\__init__.py", line 1, in &lt;module&gt;
      from nose.core import collector, main, run, run_exit, runmodule
    File "C:\Python33\lib\site-packages\nose-1.3.0-py3.3.egg\nose\core.py", line 143
      print "%s version %s" % (os.path.basename(sys.argv[0]), __version__)
                      ^
  SyntaxError: invalid syntax
  </code></pre>
  
  <p>Ok, so the Python 3.3 version of Nose contains invalid syntax for Python 3.3?</p>
  
  <p>print "%s version %s" % (os.path.basename(sys.argv[0]), <strong>version</strong>)...</p>
  
  <p>should definitely have parenthesis around it... This makes me question if nose will actually work here as it clearly looks to be made for earlier versions of Python.</p>