title: ScraPy spider crawling but not exporting
body: <p>I have a ScraPy Code that is running in shell, but when I try to export it to csv, it returns an empty file. It exports data when I do not go into a link and try to parse the description, but once I add the extra method of parsing the contents, it fails to work. Here is the code:</p>
  
  <pre><code>class MonsterSpider(CrawlSpider):
      name = "monster"
      allowed_domains = ["jobs.monster.com"]
      base_url = "http://jobs.monster.com/v-technology.aspx?"
      start_urls = [
          "http://jobs.monster.com/v-technology.aspx"
      ]
      for i in range(1,5):
          start_urls.append(base_url + "page=" + str(i))
  
      rules = (Rule(SgmlLinkExtractor(allow=("jobs.monster.com",))
           , callback = 'parse_items'),)
  
      def parse_items(self, response):
          sel = Selector(response)
          sites = sel.xpath('//div[@class="col-xs-12"]')
  
          #items = []
  
          for site in sites.xpath('.//article[@class="js_result_row"]'):
              item = MonsterItem()
              item['title'] = site.xpath('.//span[@itemprop = "title"]/text()').extract()
              item['company'] = site.xpath('.//span[@itemprop = "name"]/text()').extract()
              item['city'] = site.xpath('.//span[@itemprop = "addressLocality"]/text()').extract()
              item['state'] = site.xpath('.//span[@itemprop = "addressRegion"]/text()').extract()
              item['link'] = site.xpath('.//a[@data-m_impr_a_placement_id= "jsr"]/@href').extract()
              follow = ''.join(item["link"])
              request = Request(follow, callback = self.parse_dir_contents)
              request.meta["item"] =  item
              yield request
              #items.append(item)
              #return items
  
      def parse_dir_contents(self, response):
          item = response.meta["item"]
          item['desc'] = site.xpath('.//div[@itemprop = "description"]/text()').extract()
          return item
  </code></pre>
  
  <p>Taking out the parse_dir_contents and uncommenting the empty "lists" list and "append" code was the original code.</p>