title: How to store cumulated data after Scrapy has finished working with every URL?
body: <p>I'm trying to store some data after Scrapy has finished working (i.e after it did every URL I asked him). 
  Everytime Scrapy parse some result (through the parse function in the spider class), I append some information to an existing global object in the class itself. I would like to access that object at the end, and if possible to do everything from a Python script. Here's my spider code:</p>
  
  <pre><code>from scrapy.spider import Spider
  from scrapy.selector import Selector
  from nltk.corpus import stopwords
  
  
  from newsScrapy.items import NewsscrapyItem
  
  class newsScrapySpider(Spider):
      name = "newsScrapy"
      start_urls = []
  
      global wordMatrix
      wordMatrix = {}
  
      global prefix
      prefix = "http://www.nytimes.com/indexes/"
      sufix = "/todayspaper/index.html"
      for year in range (2000,2015):
          for month in range (1,13):
              for day in range (1,32):
                  if(month&lt;10 and day&lt;10):
                      start_urls.append (prefix+str(year)+"/"+"0"+str(month)+"/"+"0"+str(day))
                  elif (month&lt;10 and day&gt;9):
                      start_urls.append (prefix+str(year)+"/"+"0"+str(month)+"/"+str(day))
                  elif (month&gt;9 and day&lt;10):
                      start_urls.append (prefix+str(year)+"/"+str(month)+"/"+"0"+str(day))
                  else:
                      start_urls.append (prefix+str(year)+"/"+str(month)+"/"+str(day))
  
      def parse(self, response):
          sel = Selector(response)
          items = []
          text = sel.xpath('//body//text()').re('(\w+)')
  
          item = NewsscrapyItem()
  
          item['body'] = text
          item['date'] = response.url.strip(prefix)
  
          items.append(item)
  
          for word in item['body']:
              word = word.strip(' ').strip(',').strip('\n')
              word = word.lower()
              if (not word in stopwords.words('english')):
                  if(wordMatrix.__contains__((word, item['date']))):
                      wordMatrix[word,item['date']]+=1
                  else:
                      wordMatrix[word, item['date']]=1
  
  
          # print wordMatrix
          return items
  </code></pre>
  
  <p>The idea would be to access the wordMatrix variable after the end of the scraping (once every data was collected) and to do it from another Python script (for ploting per example). 
  Thanks a lot !</p>