title: Is it possible to plug a JavaScript engine with Ruby and Nokogiri?
body: <p>I'm writing an application to crawl some websites and scrape data from them. I'm using Ruby, Curl and Nokogiri to do this. In most cases it's straightforward and I only need to ping a URL and parse the HTML data. The setup works perfectly fine.</p>
  
  <p>However, in some scenarios, the websites retrieve data based on user input on some radio buttons. This invokes some JavaScript which fetches some more data from the server. The generated URL and posted data is determined by JavaScript code.</p>
  
  <p>Is it possible to use:</p>
  
  <ol>
  <li><p>A JavaScript library along with this setup which would be able to determine execute the JavaScript in the HTML page for me?</p></li>
  <li><p>Apart from using a different library, is there some integration or a way for the HTML and JS libraries to communicate? For instance if a button is clicked, Nokogiri needs to call JavaScript and then the JavaScript needs to update Nokogiri.</p></li>
  </ol>
  
  <p>In case my approach doesn't seem the best, what would your suggestion be to build a crawler + scraper on the web using Ruby.</p>
  
  <p>EDIT: Looks like point 1 is possible using therubyrace as it embeds the V8 engine in your code, but is there an alternative to 2?</p>