title: Web page scraping gems/tools available in Ruby
body: <p>I'm trying to scrape web pages in a Ruby script that I'm working on.  The purpose of the project is to show which ETFs and stock mutual funds are most compatible with the value investing philosophy.</p>
  
  <p>Some examples of pages I'd like to scrape are:</p>
  
  <pre><code>http://finance.yahoo.com/q/pr?s=SPY+Profile
  http://finance.yahoo.com/q/hl?s=SPY+Holdings
  http://www.marketwatch.com/tools/mutual-fund/list/V
  </code></pre>
  
  <p>What web scraping tools do you recommend for Ruby, and why?  Keep in mind that there are thousands of stock funds out there, so any tool I use has to be reasonably quick.</p>
  
  <p>I am new to Ruby, but I have experience using lxml to scrape web pages in Python (<a href="https://github.com/jhsu802701/dopplervalueinvesting/blob/master/screen.py" rel="noreferrer">https://github.com/jhsu802701/dopplervalueinvesting/blob/master/screen.py</a>).  Once the pages on 5000+ stocks are downloaded, lxml can scrape them all in just a few minutes.  (I remember trying BeautifulSoup but rejecting it because it was too slow.)</p>