title: VCRProxy: Record PhantomJS ajax calls with VCR inside Capybara
body: <p>I already did some research in this field, but didn't find any solution. I have a site, where asynchron ajax calls are made to facebook (using JSONP). I'm recording all my HTTP requests on the Ruby side with VCR, so I thought it would be cool, to use this feature for AJAX calls as well.</p>
  
  <p>So I played a little bit around, and came up with a proxy attempt. I'm using PhantomJS as a headless browser and poltergeist for the integration inside Capybara. Poltergeist is now configured to use a proxy like this:</p>
  
  <pre><code>  Capybara.register_driver :poltergeist_vcr do |app|
      options = {
        :phantomjs_options =&gt; [
          "--proxy=127.0.0.1:9100",
          "--proxy-type=http",
          "--ignore-ssl-errors=yes",
          "--web-security=no"
        ],
        :inspector =&gt; true
      }
      Capybara::Poltergeist::Driver.new(app, options)
    end
    Capybara.javascript_driver = :poltergeist_vcr
  </code></pre>
  
  <p>For testing purposes, I wrote a proxy server based on WEbrick, that integrates VCR:</p>
  
  <pre><code>require 'io/wait'
  require 'webrick'
  require 'webrick/httpproxy'
  
  require 'rubygems'
  require 'vcr'
  
  module WEBrick
    class VCRProxyServer &lt; HTTPProxyServer
      def service(*args)
        VCR.use_cassette('proxied') { super(*args) }
      end
    end
  end
  
  VCR.configure do |c|
    c.stub_with :webmock
    c.cassette_library_dir = '.'
    c.default_cassette_options = { :record =&gt; :new_episodes }
    c.ignore_localhost = true
  end
  
  IP   = '127.0.0.1'
  PORT = 9100
  
  reader, writer = IO.pipe
  
  @pid = fork do
    reader.close
    $stderr = writer
    server = WEBrick::VCRProxyServer.new(:BindAddress =&gt; IP, :Port =&gt; PORT)
    trap('INT') { server.shutdown }
    server.start
  end
  
  raise 'VCR Proxy did not start in 10 seconds' unless reader.wait(10)
  </code></pre>
  
  <p>This works well with every localhost call, and they get well recorded. The HTML, JS and CSS files are recorded by VCR. Then I enabled the <code>c.ignore_localhost = true</code> option, cause it's useless (in my opinion) to record localhost calls.</p>
  
  <p>Then I tried again, but I had to figure out, that the AJAX calls that are made on the page aren't recorded. Even worse, they doesn't work inside the tests anymore.</p>
  
  <p>So to come to the point, my question is: Why are all calls to JS files on the localhost recorded, and JSONP calls to external ressources not? It can't be the jsonP thing, cause it's a "normal" ajax request. Or is there a bug inside phantomjs, that AJAX calls aren't proxied? If so, how could we fix that?</p>
  
  <p>If it's running, I want to integrate the start and stop procedure inside</p>
  
  <p><strong>------- UPDATE -------</strong></p>
  
  <p>I did some research and came to the following point: the proxy has some problems with HTTPS calls and binary data through HTTPS calls.</p>
  
  <p>I started the server, and made some curl calls:</p>
  
  <pre><code>curl --proxy 127.0.0.1:9100 http://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png
  </code></pre>
  
  <p>This call gets recorded as it should. The request and response output from the proxy is</p>
  
  <pre><code>GET http://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png HTTP/1.1
  User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
  Host: d3jgo56a5b0my0.cloudfront.net
  Accept: */*
  Proxy-Connection: Keep-Alive
  
  HTTP/1.1 200 OK 
  Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12)
  Date: Tue, 20 Nov 2012 10:13:10 GMT
  Content-Length: 0
  Connection: Keep-Alive
  </code></pre>
  
  <p>But this call doesn't gets recorded, there must be some problem with HTTPS:</p>
  
  <pre><code>curl --proxy 127.0.0.1:9100 https://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png
  </code></pre>
  
  <p>The header output is:</p>
  
  <pre><code>CONNECT d3jgo56a5b0my0.cloudfront.net:443 HTTP/1.1
  Host: d3jgo56a5b0my0.cloudfront.net:443
  User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
  Proxy-Connection: Keep-Alive
  
  HTTP/1.1 200 OK 
  Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12)
  Date: Tue, 20 Nov 2012 10:15:48 GMT
  Content-Length: 0
  Connection: close
  </code></pre>
  
  <p>So, I thought maybe the proxy can't handle HTTPS, but it can (as long as I'm getting the output on the console after the cURL call). Then I thought, maybe VCR can't mock HTTPS requests. But using this script, VCR mocks out HTTPS requests, when I don't use it inside the proxy:</p>
  
  <pre><code>require 'vcr'
  
  VCR.configure do |c|
    c.hook_into :webmock
    c.cassette_library_dir = 'cassettes'
  end
  
  uri = URI("https://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png")
  
  VCR.use_cassette('https', :record =&gt; :new_episodes) do
    http = Net::HTTP.new(uri.host, uri.port)
    http.use_ssl = true
    http.verify_mode = OpenSSL::SSL::VERIFY_NONE
    response = http.request_get(uri.path)
    puts response.body
  end
  </code></pre>
  
  <p>So what is the problem? VCR handles HTTPS and the proxy handles HTTPS. Why they don't play together?</p>