title: Locally run all of the spiders in Scrapy
body: <p>Is there a way to run all of the spiders in a Scrapy project without using the Scrapy daemon? There used to be a way to run multiple spiders with <code>scrapy crawl</code>, but that syntax was removed and Scrapy's code changed quite a bit.</p>
  
  <p>I tried creating my own command:</p>
  
  <pre><code>from scrapy.command import ScrapyCommand
  from scrapy.utils.misc import load_object
  from scrapy.conf import settings
  
  class Command(ScrapyCommand):
      requires_project = True
  
      def syntax(self):
          return '[options]'
  
      def short_desc(self):
          return 'Runs all of the spiders'
  
      def run(self, args, opts):
          spman_cls = load_object(settings['SPIDER_MANAGER_CLASS'])
          spiders = spman_cls.from_settings(settings)
  
          for spider_name in spiders.list():
              spider = self.crawler.spiders.create(spider_name)
              self.crawler.crawl(spider)
  
          self.crawler.start()
  </code></pre>
  
  <p>But once a spider is registered with <code>self.crawler.crawl()</code>, I get assertion errors for all of the other spiders:</p>
  
  <pre><code>Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/scrapy/cmdline.py", line 138, in _run_command
      cmd.run(args, opts)
    File "/home/blender/Projects/scrapers/store_crawler/store_crawler/commands/crawlall.py", line 22, in run
      self.crawler.crawl(spider)
    File "/usr/lib/python2.7/site-packages/scrapy/crawler.py", line 47, in crawl
      return self.engine.open_spider(spider, requests)
    File "/usr/lib/python2.7/site-packages/twisted/internet/defer.py", line 1214, in unwindGenerator
      return _inlineCallbacks(None, gen, Deferred())
  --- &lt;exception caught here&gt; ---
    File "/usr/lib/python2.7/site-packages/twisted/internet/defer.py", line 1071, in _inlineCallbacks
      result = g.send(result)
    File "/usr/lib/python2.7/site-packages/scrapy/core/engine.py", line 215, in open_spider
      spider.name
  exceptions.AssertionError: No free spider slots when opening 'spidername'
  </code></pre>
  
  <p>Is there any way to do this? I'd rather not start subclassing core Scrapy components just to run all of my spiders like this.</p>