title: Handle pagination in python scrapy
body: <p>I am using scrapy python to scrape a particular site. The site has pagination of the form below:</p>
  
  <p><a href="http://www.example.com/s/ref=lp_1805560031_pg_4?rh=n%3A976419031%2Cn%3A%21976420031%2Cn%3A1389401031%2Cn%3A1389432031%2Cn%3A1805560031&amp;" rel="nofollow">http://www.example.com/s/ref=lp_1805560031_pg_4?rh=n%3A976419031%2Cn%3A%21976420031%2Cn%3A1389401031%2Cn%3A1389432031%2Cn%3A1805560031&amp;</a><strong>page=4</strong>&amp;ie=UTF8&amp;qid=1400668237</p>
  
  <p>How can i include handle the pagination in this case if i want to scrape from say page 1 to page 30;</p>
  
  <p>I tried this :</p>
  
  <pre><code>class MySpider(BaseSpider):
      start_urls = ['http://www.example.com/s/ref=lp_1805560031_pg_4?rh=n%3A976419031%2Cn%3A%21976420031%2Cn%3A1389401031%2Cn%3A1389432031%2Cn%3A1805560031&amp;page=%s&amp;ie=UTF8&amp;qid=1400668237' % page for page in xrange(1,30)]
  </code></pre>
  
  <p>But its not working</p>
  
  <p><strong>EDIT :</strong>  Am using domain as example.com just for the question purpose</p>