 <p>You might be interested in looking at <a href="http://nrabinowitz.github.com/pjscrape/" rel="nofollow">Pjscrape</a> (disclaimer: this is my project). It's a web-scraping tool built on PhantomJS, giving you full jQuery access to the page in a headless Webkit browser context. It makes it very easy to pull semi-structured data from webpages via the command line, particularly if the page you're scraping has a consistent structure for new elements.</p> <p>For example, you can pull all the course titles from <a href="http://www.ischool.berkeley.edu/courses/catalog" rel="nofollow">this course catalog</a> with the following code:</p> <pre><code>pjs.addScraper( // the page you're scraping 'http://www.ischool.berkeley.edu/courses/catalog', // selector for elements you want to pull text from '.views-row .views-field-title' ); // suppress STDOUT logging pjs.config('log', 'none'); </code></pre> <p>Running this from the command line gives you JSON to STDOUT by default:</p> <pre><code>~&gt; phantomjs /path/to/pjscrape.js my_script.js ["W10. Introduction to Information","24. Freshman Seminar", ...] </code></pre> <p>So it would be pretty simple to run this script on a regular basis, capture the output in a file, and then alert you when the new output doesn't match the previous scrape. You can also write your own scraper functions, so there's a lot of flexibility for more complex scraping if a simple selector won't do the trick.</p>