title: AppEngine urlfetch validate_certificate=False/None not being respected
body: <p>In the AppEngine developer appserver I am getting an error like this:</p>
  
  <pre><code>SSLCertificateError: Invalid and/or missing SSL certificate for URL ...
  </code></pre>
  
  <p>when I am making a fetch like this to an <code>https</code> server with a self-signed certificate (almost always <code>localhost</code> port-forwarded over ssh to a vm):</p>
  
  <pre><code>result = urlfetch.fetch(url=url, method=method, payload=payload,
                          deadline=DEADLINE, validate_certificate=None)
  </code></pre>
  
  <p>One would not expect SSL failures for invalid certificates where <code>validate_certificate</code> is <code>False</code>, though this is quite possibly a side-effect of the 2.7.9 policy in Python to always validate ssl certificates.</p>
  
  <p>Note that passing <code>False</code> (instead of <code>None</code>) for <code>validate_certificate</code> does not work either.</p>
  
  <p>This problem happens on Python 2.7.9-10 via Homebrew/XCode on OS X 10.10.2-4 with AppEngine 1.9.18 through 1.19.26.</p>
  
  <p>There are issues (e.g. <a href="https://code.google.com/p/googleappengine/issues/detail?id=12096" rel="nofollow noreferrer">12096</a>) about this on Google App Engine, but I am looking for a workaround.</p>
  
  <p>Here's what I've tried to work around this:</p>
  
  <ol>
  <li><p>Add the certificate to the Mac's login keychain  (works in the browser, not from Python)</p></li>
  <li><p>Add the certificate to <code>app-engine-python/lib/cacerts/cacerts.txt</code> and/or <code>./lib/cacerts/urlfetch_cacerts.txt</code>  (though this probably requires turning verification <em>on</em> for it to work, since that appears to be the only case where they are used) with e.g.</p>
  
  <blockquote>
    <p>$ echo >> /usr/local/share/app-engine-python/lib/cacerts/urlfetch_cacerts.txt</p>
    
    <p>$ openssl x509 -subject -in server.crt >> /usr/local/share/app-engine-python/lib/cacerts/urlfetch_cacerts.txt</p>
  </blockquote></li>
  <li><p>Disable ssl HTTPs checking with the <a href="https://www.python.org/dev/peps/pep-0476/" rel="nofollow noreferrer">PEP-0476</a> workaround i.e.</p>
  
  <p><code>ssl._create_default_https_context = ssl._create_unverified_context</code></p>
  
  <p>at or after <code>import ssl</code> (around line 1149) of <a href="https://github.com/GoogleCloudPlatform/appengine-python-vm-runtime/blob/master/python_vm_runtime/google/appengine/dist27/python_std_lib/httplib.py#L1149" rel="nofollow noreferrer"><code>google/appengine/dist27/python_std_lib/httplib.py</code></a></p></li>
  </ol>
  
  <p>This is particularly problematic on Mac since downgrading as of XCode 7/OS X El Capital is no longer a practical option.</p>
  
  <p>A preferable workaround would not involve monkey-patching the AppEngine code proper every time the development appserver is updated.</p>
  
  <hr>
  
  <p><strong>EDIT</strong></p>
  
  <p>Note that the Mac builtin OpenSSL certificates are stored in <code>/System/Library/OpenSSL</code>, which is protected with <a href="https://apple.stackexchange.com/a/193379/5522">SIP/rootlessness</a>, which frankly is a pain to muck with and a worthwhile feature to keep if we can.</p>
  
  <p>I have verified that the certificate validates by using <code>openssl s_client -connect localhost:7500 -CAfile server.pem</code>.</p>
  
  <p>It's been added to the Keychain and to <code>/usr/local/etc/openssl/certs</code> with the <code>hash.#</code> format where the hash comes from <code>openssl x509 -subject_hash -in server.pem</code> (or the homebrew ssl, namely <code>/usr/local/Cellar/openssl/1.0.2d_1/bin/openssl</code>).  In which case <code>/usr/local/Cellar/openssl/1.0.2d_1/bin/openssl s_client -connect localhost:7500</code> verifies the certificate (but python still does not).</p>
  
  <p>I have tried using the homebrew version of python and openssl, but to no avail.  Running the following in Python seems to always fail;</p>
  
  <pre><code>./pve/bin/python -c "import requests; requests.get('https://localhost:7500')"
  </code></pre>
  
  <p>This also fails where <code>SSL_CERT_FILE</code> is set to the server's certificate (i.e. for added measure one might expect it to work since the <code>openssl</code> command essentially works like this), and also fails where <code>SSL_CERT_PATH</code> is set to <code>/usr/local/etc/openssl/certs</code>.</p>
  
  <p>Note, <code>pve</code> is a virtual env where <code>help(ssl)</code> shows a <code>FILE</code> of <code>/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py</code></p>
  
  <p>Further verifying that homebrew Python's <code>_ssl.so</code> links to homebrew's openssl I ran:</p>
  
  <p>xcrun otool -L 
  /usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_ssl.so</p>
  
  <p>which returns</p>
  
  <blockquote>
    <p>./Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_ssl.so:</p>
    
    <p>/usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)</p>
    
    <p>/usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)</p>
    
    <p>/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1225.1.1)</p>
  </blockquote>
  
  <p>If one runs <code>brew info openssl</code> it remarks under <code>CAVEATS</code>:</p>
  
  <blockquote>
    <p>A CA file has been bootstrapped using certificates from the system
    keychain. To add additional certificates, place .pem files in
      /usr/local/etc/openssl/certs</p>
  </blockquote>
  
  <p>but clearly for some reason python is not using homebrew's openssl algorithm for finding certificates.</p>
  
  <p>So I remain at a loss as to why Python standard library is not validating certificates that are in the OpenSSL directory specified in the documents as well as the Keychain (in both <code>.pem</code> and <code>.p12</code> formats, with "always trust" for <code>Secure Sockets Layer (SSL)</code>).</p>