title: Python not finding its Packages
body: <p>I wrote myself a handy bash script, which solves the task of creating a <code>virtualenv</code> with its own compiled virtualenv and python. It aims at creating a mostly self contained virtualenv, with maybe only native libraries installed in system level if necessary, but installing all python packages and virtualenv and pip and such things inside the virtualenv.</p>
  
  <p>The script can be found <a href="https://github.com/ZelphirKaltstahl/bash-scripts/blob/master/self_contained_venv.sh" rel="nofollow">here</a>.</p>
  
  <p>I invoke the script as follows:</p>
  
  <pre><code>self_contained_venv.sh \
  -n udacity_model_building_and_validation \
  -p https://www.python.org/ftp/python/3.4.4/Python-3.4.4.tar.xz \
  -v https://pypi.python.org/packages/c8/82/7c1eb879dea5725fae239070b48187de74a8eb06b63d9087cd0a60436353/virtualenv-15.0.1.tar.gz#md5=28d76a0d9cbd5dc42046dd14e76a6ecc \
  -d pandas scikit-learn seaborn
  </code></pre>
  
  <p>Given the required packages for compiling python and virtualenv are installed on the system, the script creates a nice virtualenv. However, when I try to access any installed modules/packages from within the virtualenv, python is not able to find them. To demonstrate this, I'll put some output of commands and code here:</p>
  
  <p>First of all of course I have to activate the virtualenv:</p>
  
  <pre><code>. bin/activate
  </code></pre>
  
  <p><strong>output</strong>: None, works without problem.</p>
  
  <p>Then I print the pythonpath python is aware of:</p>
  
  <pre><code>import sys
  for i in sys.path:
      print(i)
  </code></pre>
  
  <p><strong>output</strong>:</p>
  
  <pre><code>/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python34.zip
  /home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4
  /home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/plat-linux
  /home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/lib-dynload
  /home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4
  /home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/plat-linux
  /home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
  </code></pre>
  
  <p>So far so good. Then I try to import a module / package I installed during usage of my bash script: <code>pandas</code>:</p>
  
  <pre><code>python
  </code></pre>
  
  <p>(IDLE is running)</p>
  
  <pre><code>import pandas as pd
  </code></pre>
  
  <p><strong>output</strong>:</p>
  
  <pre><code>Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  ImportError: No module named 'pandas'
  </code></pre>
  
  <p>Another try:</p>
  
  <pre><code>import numpy as np
  </code></pre>
  
  <p><strong>output</strong>:</p>
  
  <pre><code>Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  ImportError: No module named 'numpy'
  </code></pre>
  
  <p>Huh? So none of the packages is available? Lets check pip again:</p>
  
  <pre><code>which pip
  </code></pre>
  
  <p><strong>output</strong>:</p>
  
  <pre><code>alias pip='localpython/bin/pip3.4'
  ./localpython/bin/pip3.4
  </code></pre>
  
  <p>Ok, so it's using my local pip.</p>
  
  <p>Check packages:</p>
  
  <pre><code>pip list
  </code></pre>
  
  <p><strong>output</strong>:</p>
  
  <pre><code>numpy (1.11.0)
  pandas (0.18.1)
  pip (8.1.2)
  psutil (4.1.0)
  Python-contrib-nbextensions (alpha)
  python-dateutil (2.5.3)
  pytz (2016.4)
  PyYAML (3.11)
  setuptools (18.2)
  six (1.10.0)
  virtualenv (15.0.1)
  </code></pre>
  
  <p>Hm the packages are there, so why can't python find them? Let's see where those packages are located, simply by trying to remove one:</p>
  
  <pre><code>pip uninstall pandas
  </code></pre>
  
  <p><strong>output</strong> (shortened, because it fills many pages):</p>
  
  <pre><code>Uninstalling pandas-0.18.1:
   /home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas-0.18.1-py3.4.egg-info
    /home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__init__.py
    /home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/__init__.cpython-34.pyc
    /home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/_version.cpython-34.pyc
    /home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/info.cpython-34.pyc
  </code></pre>
  
  <p>Aha, so the packages <em>are</em> in the path.</p>
  
  <p>Another attempt on finding out if python looks in the right places:</p>
  
  <pre><code>&gt;&gt;&gt; from distutils.sysconfig import get_python_lib
  &gt;&gt;&gt; print(get_python_lib())
  /home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
  </code></pre>
  
  <p>So this one indicates it does not look in the right place, if I understand correctly.</p>
  
  <p><strong>Note</strong>:</p>
  
  <p>The script does the following things, which might not be obvious:</p>
  
  <ul>
  <li>compile python with a prefix (local python)</li>
  <li>compile virtualenv for the local python</li>
  <li>it aliases pip to the local pip of the version of the installed python</li>
  <li>it aliases virtualenv to the locally installed one</li>
  <li>it installs packages from pypi if specified</li>
  <li>it updates the local pip if there is a newer version available</li>
  </ul>
  
  <p>I am a beginner still at writing bash scripts, so I think the structure and logic of the script is fairly easy to understand. It also prints information about success of its operations in the terminal.</p>
  
  <hr>
  
  <p><strong>Further Notes</strong>:</p>
  
  <ul>
  <li>I did not use <code>su</code> or <code>sudo</code> to run the script.</li>
  <li>My OS is a Fedora 22, although I think in this case any major distro would work the same way.</li>
  <li>OS has been updated recently.</li>
  </ul>
  
  <p><strong>Question</strong>: So why can't Python find them / its own packages? (What do I need to change?)</p>