title: Apache SetEnv not working as expected with mod_wsgi
body: <p>In a flask application I wrote, I make use of an external library which can be configured using environment variables. Note: I wrote this external library myself. So I <em>could</em> make changes if necessary. When running from the command line an running the flask server with:</p>
  
  <pre><code># env = python virtual environment
  ENV_VAR=foo ./env/bin/python myapp/webui.py
  </code></pre>
  
  <p>it all woks as expected. But after deploying it to apache, and using <code>SetEnv</code> it does <em>not</em> work anymore. In fact, printing out <code>os.environ</code> to <code>stderr</code> (so it shows up in the apache logs reveals, that the <code>wsgi</code> process seems to be in a very different environment (for one, <code>os.environ['PWD']</code> seems to be <em>way</em> off. In fact, it points to my development folder.</p>
  
  <p>To help identifying the problem, following are the relevant parts of the application as a standalone hello-world app. The error output, and observations are at the very end of the post.</p>
  
  <h3>App folder layout:</h3>
  
  <p>Python app:</p>
  
  <pre><code>.
  ├── myapp.ini
  ├── setup.py
  └── testenv
      ├── __init__.py
      ├── model
      │   └── __init__.py
      └── webui.py
  </code></pre>
  
  <p>Apache folder (<code>/var/www/michel/testenv</code>):</p>
  
  <pre><code>.
  ├── env
  │   ├── [...]
  ├── logs
  │   ├── access.log
  │   └── error.log
  └── wsgi
  └── app.wsgi
  </code></pre>
  
  <h3>myapp.ini</h3>
  
  <pre><code>[app]
  somevar=somevalue
  </code></pre>
  
  <h3>setup.py</h3>
  
  <pre><code>from setuptools import setup, find_packages
  
  setup(
      name="testenv",
      version='1.0dev1',
      description="A test app",
      long_description="Hello World!",
      author="Some Author",
      author_email="author@example.com",
      license="BSD",
      include_package_data=True,
      install_requires = [
        'flask',
        ],
      packages=find_packages(exclude=["tests.*", "tests"]),
      zip_safe=False,
  )
  </code></pre>
  
  <h3>testenv/<strong>init</strong>.py</h3>
  
  <pre><code># empty
  </code></pre>
  
  <h3>testenv/model/<strong>init</strong>.py</h3>
  
  <pre><code>from os.path import expanduser, join, exists
  from os import getcwd, getenv, pathsep
  import logging
  import sys
  
  __version__ = '1.0dev1'
  
  LOG = logging.getLogger(__name__)
  
  def find_config():
      """
      Searches for an appropriate config file. If found, return the filename, and
      the parsed search path
      """
  
      path = [getcwd(), expanduser('~/.mycompany/myapp'), '/etc/mycompany/myapp']
      env_path = getenv("MYAPP_PATH")
      config_filename = getenv("MYAPP_CONFIG", "myapp.ini")
      if env_path:
          path = env_path.split(pathsep)
  
      detected_conf = None
      for dir in path:
          conf_name = join(dir, config_filename)
          if exists(conf_name):
              detected_conf = conf_name
              break
      return detected_conf, path
  
  def load_config():
      """
      Load the config file.
      Raises an OSError if no file was found.
      """
      from ConfigParser import SafeConfigParser
  
      conf, path = find_config()
      if not conf:
          raise OSError("No config file found! Search path was %r" % path)
  
      parser = SafeConfigParser()
      parser.read(conf)
      LOG.info("Loaded settings from %r" % conf)
      return parser
  
  try:
      CONF = load_config()
  except OSError, ex:
      # Give a helpful message instead of a scary stack-trace
      print &gt;&gt;sys.stderr, str(ex)
      sys.exit(1)
  </code></pre>
  
  <h3>testenv/webui.py</h3>
  
  <pre><code>from testenv.model import CONF
  
  from flask import Flask
  
  app = Flask(__name__)
  
  @app.route('/')
  def index():
      return "Hello World %s!" % CONF.get('app', 'somevar')
  
  if __name__ == '__main__':
      app.debue = True
      app.run()
  </code></pre>
  
  <h3>Apache config</h3>
  
  <pre><code>&lt;VirtualHost *:80&gt;
      ServerName testenv-test.my.fq.dn
      ServerAlias testenv-test
  
      WSGIDaemonProcess testenv user=michel threads=5
      WSGIScriptAlias / /var/www/michel/testenv/wsgi/app.wsgi
      SetEnv MYAPP_PATH /var/www/michel/testenv/config
  
      &lt;Directory /var/www/michel/testenv/wsgi&gt;
          WSGIProcessGroup testenv
          WSGIApplicationGroup %{GLOBAL}
          Order deny,allow
          Allow from all
      &lt;/Directory&gt;
  
      ErrorLog /var/www/michel/testenv/logs/error.log
      LogLevel warn
  
      CustomLog /var/www/michel/testenv/logs/access.log combined
  
  &lt;/VirtualHost&gt;
  </code></pre>
  
  <h3>app.wsgi</h3>
  
  <pre><code>activate_this = '/var/www/michel/testenv/env/bin/activate_this.py'
  execfile(activate_this, dict(__file__=activate_this))
  
  from os import getcwd
  import logging, sys
  
  from testenv.webui import app as application
  
  # You may want to change this if you are using another logging setup
  logging.basicConfig(stream=sys.stderr, level=logging.DEBUG)
  
  LOG = logging.getLogger(__name__)
  LOG.debug('Current path: {0}'.format(getcwd()))
  
  # Application config
  application.debug = False
  
  # vim: set ft=python :
  </code></pre>
  
  <h1>Error and observations</h1>
  
  <p>This is the output of the apache error log.</p>
  
  <pre><code>[Thu Jan 26 10:48:15 2012] [error] No config file found! Search path was ['/home/users/michel', '/home/users/michel/.mycompany/myapp', '/etc/mycompany/myapp']
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101] mod_wsgi (pid=17946): Target WSGI script '/var/www/michel/testenv/wsgi/app.wsgi' cannot be loaded as Python module.
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101] mod_wsgi (pid=17946): SystemExit exception raised by WSGI script '/var/www/michel/testenv/wsgi/app.wsgi' ignored.
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101] Traceback (most recent call last):
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101]   File "/var/www/michel/testenv/wsgi/app.wsgi", line 10, in &lt;module&gt;
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101]     from testenv.webui import app as application
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101]   File "/var/www/michel/testenv/env/lib/python2.6/site-packages/testenv-1.0dev1-py2.6.egg/testenv/webui.py", line 1, in &lt;module&gt;
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101]     from testenv.model import CONF
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101]   File "/var/www/michel/testenv/env/lib/python2.6/site-packages/testenv-1.0dev1-py2.6.egg/testenv/model/__init__.py", line 51, in &lt;module&gt;
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101]     sys.exit(1)
  [Thu Jan 26 10:48:15 2012] [error] [client 10.115.192.101] SystemExit: 1
  </code></pre>
  
  <p>My first observation is that the environment variable <code>MYAPP_PATH</code> does not appear in <code>os.environ</code> (this is not visible in this output, but I tested it, and it's not there!). As such, the config "resolver" falls back to the default path.</p>
  
  <p>And my second observation is the search path for the config file lists <code>/home/users/michel</code> as return value of <code>os.getcwd()</code>. I was actually expecting something inside <code>/var/www/michel/testenv</code>.</p>
  
  <p>My instinct tells me, that the way I am doing config resolution is not right. Mainly because code is executed at import-time. This leads me to the idea, that maybe the config-resolution code is executed <em>before</em> the WSGI environment is properly set up. Am I onto something there?</p>
  
  <h2>Short discussion / tangential question</h2>
  
  <p>How would <em>you</em> do the config resolution in this case? Given that the "model" sub-folder is in reality an external module, which should also work in non-wsgi applications, and should provide a method to configure a database connection.</p>
  
  <p>Personally, I like the way I search for config files while still being able to override it. Only, the fact that code is executed at import-time is making my spider-senses tingling like crazy. The rationale behind this: Configuration handling is completely hidden (abstraction-barrier) by fellow developers using this module, and it "just works". They just need to import the module (with an existing config-file of course) and can jump right in without knowing any DB details. This also gives them an easy way to work with different databases (dev/test/deployment) and switch between them easily.</p>
  
  <p>Now, inside mod_wsgi it does not anymore :(</p>
  
  <h3>Update:</h3>
  
  <p>Just now, to test my above idea, I changed the <code>webui.py</code> to the following:</p>
  
  <pre><code>import os
  
  from flask import Flask, jsonify
  
  app = Flask(__name__)
  
  @app.route('/')
  def index():
      return jsonify(os.environ)
  
  if __name__ == '__main__':
      app.debue = True
      app.run()
  </code></pre>
  
  <p>The output on the webpage is the following:</p>
  
  <pre><code>{
      LANG: "C",
      APACHE_RUN_USER: "www-data",
      APACHE_PID_FILE: "/var/run/apache2.pid",
      PWD: "/home/users/michel/tmp/testenv",
      APACHE_RUN_GROUP: "www-data",
      PATH: "/usr/local/bin:/usr/bin:/bin",
      HOME: "/home/users/michel/"
  }
  </code></pre>
  
  <p>This shows the same environment as the one seen by other debugging methods. So my initial though was wrong. But now I realised something stranger yet. <code>os.environment['PWD']</code> is set to the folder where I have my development files. This is not at <em>all</em> where the application is running. Stranger yet, <code>os.getcwd()</code> returns <code>/home/users/michel</code>? This is inconsistent with what I see in <code>os.environ</code>. Should it not be the same as <code>os.environ['PWD']</code>?</p>
  
  <p>The most important problem remains though: Why is the value set by apache's <code>SetEnv</code> (<code>MYAPP_PATH</code> in this case) not found in <code>os.environ</code>?</p>