title: Varnish/Nginx cached SSL Certificate mystery
body: <p>I have Varnish load balancing three front end Rails servers with Nginx acting as a reverse proxy for FastCGI workers. Yesterday, our certificate expired, and I got a new certificate from GoDaddy, and installed it. When accessing static resources directly, I see the updated certificate, but when accessing them from a "virtual subdomain" I'm seeing the old certificate. My nginx config only cites my new chained certificate, so I'm wondering how the old certificate is being displayed. I've even removed it from the directory.</p>
  
  <p>example:
  <a href="https://www212.doostang.com/javascripts/base_packaged.js?1331831461" rel="nofollow">https://www212.doostang.com/javascripts/base_packaged.js?1331831461</a> (no certificate problem with SSL)
  <a href="https://asset5.doostang.com/javascripts/base_packaged.js?1331831461" rel="nofollow">https://asset5.doostang.com/javascripts/base_packaged.js?1331831461</a> (the old certificate is being used!) (maps to www212.doostang.com)</p>
  
  <ul>
  <li>I've reloaded and even stopped-and-restarted nginx, tested nginx to make sure that it's reading from the right config, and restarted varnish with a new cache file.</li>
  <li><p>When I curl the file at asset5.doostang.com I get a certificate error:</p>
  
  <p>curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
  error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
  More details here: <a href="http://curl.haxx.se/docs/sslcerts.html" rel="nofollow">http://curl.haxx.se/docs/sslcerts.html</a></p>
  
  <p>curl performs SSL certificate verification by default, using a "bundle"
   of Certificate Authority (CA) public keys (CA certs). If the default
   bundle file isn't adequate, you can specify an alternate file
   using the --cacert option.
  If this HTTPS server uses a certificate signed by a CA represented in
   the bundle, the certificate verification probably failed due to a
   problem with the certificate (it might be expired, or the name might
   not match the domain name in the URL).
  If you'd like to turn off curl's verification of the certificate, use
   the -k (or --insecure) option.</p></li>
  <li><p>When I add the -k option, I get the file requested, and I can see it in my nginx access log. I don't get an nginx error when I don't provide the -k; nginx is silent about the certificate error.</p></li>
  </ul>
  
  <blockquote>
    <p>10.99.110.27 - - [20/Apr/2012:18:02:52 -0700] "GET /javascripts/base_packaged.js?1331831461 HTTP/1.0" 200 5740 "-"
    "curl/7.21.3 (x86_64-pc-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8o
    zlib/1.2.3.4 libidn/1.18"</p>
  </blockquote>
  
  <p>I've put what I think is the relevant part of the nginx config, below:</p>
  
  <pre><code>server {
    # port to listen on. Can also be set to an IP:PORT
    listen 443;
    server_name www.doostang.com, *.doostang.com;
  
    passenger_enabled on;
    rails_env production;
  
    ssl on;
    ssl_certificate /.../doostang_combined.crt;
    ssl_certificate_key /.../doostang.com.key;
    ssl_protocols SSLv3;
  
    # doc root
    root /.../public/files;
  
    if ($host = 'doostang.com' ) {
          rewrite  ^/(.*)$  https://www.doostang.com/$1  permanent;
     }
  }
  
  
  # Catchall redirect
  server {
    # port to listen on. Can also be set to an IP:PORT
    listen 443;
  
    ssl on;
    ssl_certificate /.../doostang_combined.crt;
    ssl_certificate_key /.../doostang.com.key;
  
    rewrite ^(.*)$ https://www.doostang.com$1;
  } 
  </code></pre>