title: Confused with Rules in Scrapy in python spider
body: <p>I have one doubt in scrapy spider. Suppose i ahve this code</p>
  
  <pre><code> name = 'myspider'
      allowed_domains = ['domain.com']
      start_urls = ['http://www.domain.com/foo/']
  
      rules = (
          Rule(SgmlLinkExtractor(allow=r'-\w+.html$'), callback='parse_item', follow=True),
      )
  
      def parse_item(self, response):
          hxs = HtmlXPathSelector(response)
  </code></pre>
  
  <p>I want to know that will the spider first go to the start url and parse the page and then use rules for extracting  links</p>
  
  <p>or spider don't parse the first page but will start with rules</p>
  
  <p>I have seen that if my rules don't match then i don't get any results but at least don't it should have parsed the starting page</p>