title: How to get scraped items from main script using scrapy?
body: <p>I hope to get a list of scraped items in main script instead of using scrapy shell. </p>
  
  <p>I know there is a method <code>parse</code> in class <code>FooSpider</code> I define, and this method return a list of <code>Item</code>. Scrapy framework calls this method. But, how can I get this returned list by myself?</p>
  
  <p>I found so many posts about that, but I don't understand what they were saying. </p>
  
  <p>As a context, I put official example code here</p>
  
  <pre><code>import scrapy
  
  from tutorial.items import DmozItem
  
  class DmozSpider(scrapy.Spider):
      name = "dmoz"
      allowed_domains = ["dmoz.org"]
      start_urls = [
          "http://www.dmoz.org/Computers/Programming/Languages/Python/",
      ]
  
      def parse(self, response):
          for href in response.css("ul.directory.dir-col &gt; li &gt; a::attr('href')"):
              url = response.urljoin(href.extract())
              yield scrapy.Request(url, callback=self.parse_dir_contents)
  
      def parse_dir_contents(self, response):
          result = []
          for sel in response.xpath('//ul/li'):
              item = DmozItem()
              item['title'] = sel.xpath('a/text()').extract()
              item['link'] = sel.xpath('a/@href').extract()
              item['desc'] = sel.xpath('text()').extract()
              result.append(item)
  
          return result
  </code></pre>
  
  <p>How could I get returned <code>result</code> from a main python script like <code>main.py</code> or <code>run.py</code>?</p>
  
  <pre><code>if __name__ == "__main__":
      ...
      result = xxxx()
      for item in result:
          print item
  </code></pre>
  
  <p>Could anyone provide a code snippet in which I get this returned <code>list</code> from somewhere?</p>
  
  <p>Thank you very much!</p>