title: Scrapy spider not terminating with use of CloseSpider extension
body: <p>I have set up a Scrapy spider that parses an xml feed, processing some 20,000 records.</p>
  
  <p>For the purposes of development, I'd like to limit the number of items processed. From reading the Scrapy docs I identified I need to use the <a href="https://doc.scrapy.org/en/latest/topics/extensions.html#module-scrapy.extensions.closespider" rel="nofollow noreferrer">CloseSpider</a> extension.</p>
  
  <p>I have followed the guide on how to enable this - in my spider config I have the following:</p>
  
  <pre><code>CLOSESPIDER_ITEMCOUNT = 1
  EXTENSIONS = {
      'scrapy.extensions.closespider.CloseSpider': 500,
  }
  </code></pre>
  
  <p>However, my spider never terminates - I'm aware that the <code>CONCURRENT_REQUESTS</code> setting affects when the spider actually terminates (as it will carry on processing each concurrent request), but this is only set to the default of 16, and yet my spider will continue to process all the items.</p>
  
  <p>I've tried using the <code>CLOSESPIDER_TIMEOUT</code> setting instead, but similarly this has no effect.</p>
  
  <p>Here is some debug info, from when I run the spider:</p>
  
  <pre><code>2017-06-15 12:14:11 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: myscraper)
  2017-06-15 12:14:11 [scrapy.utils.log] INFO: Overridden settings: {'BOT_NAME': 'myscraper', 'CLOSESPIDER_ITEMCOUNT': 1, 'FEED_URI': 'file:///tmp/myscraper/export.jsonl', 'NEWSPIDER_MODULE': 'myscraper.spiders', 'ROBOTSTXT_OBEY': True, 'SPIDER_MODULES': ['myscraper.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36'}
  2017-06-15 12:14:11 [scrapy.middleware] INFO: Enabled extensions:
  ['scrapy.extensions.corestats.CoreStats',
   'scrapy.extensions.telnet.TelnetConsole',
   'scrapy.extensions.feedexport.FeedExporter',
   'scrapy.extensions.logstats.LogStats',
   'scrapy.extensions.closespider.CloseSpider']
  2017-06-15 12:14:11 [scrapy.middleware] INFO: Enabled downloader middlewares:
  ['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
   'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
   'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
   'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
   'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
   'scrapy.downloadermiddlewares.retry.RetryMiddleware',
   'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
   'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
   'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
   'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
   'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
   'scrapy.downloadermiddlewares.stats.DownloaderStats']
  2017-06-15 12:14:11 [scrapy.middleware] INFO: Enabled spider middlewares:
  ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
   'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
   'scrapy.spidermiddlewares.referer.RefererMiddleware',
   'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
   'scrapy.spidermiddlewares.depth.DepthMiddleware']
  2017-06-15 12:14:11 [scrapy.middleware] INFO: Enabled item pipelines:
  ['myscraper.pipelines.MyScraperPipeline']
  2017-06-15 12:14:11 [scrapy.core.engine] INFO: Spider opened
  </code></pre>
  
  <p>As can be seen, the <code>CloseSpider</code> extension and the <code>CLOSESPIDER_ITEMCOUNT</code> settings are being applied.</p>
  
  <p>Any ideas why this is not working?</p>