title: Scrapy and Selenium StaleElementReferenceException
body: <p>There is several click-able elements on the page and I'm trying to scrape some pages behind, but I have this error and spider closed after first click:</p>
  
  <pre><code>StaleElementReferenceException: Message: Element not found in the cache - perhaps the page has changed since it was looked up
  </code></pre>
  
  <p>For now I just trying to get page opened to catch new url. Here is my code</p>
  
  <pre><code>from scrapy import signals
  from scrapy.http import TextResponse
  from scrapy.spider import Spider
  from scrapy.selector import Selector
  from scrapy.xlib.pydispatch import dispatcher
  
  from MySpider.items import MyItem
  
  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  
  import time
  
  class MySpider(Spider):
      name = "myspider"
      allowed_domains = ["http://example.com"]
      base_url = 'http://example.com'
      start_urls = ["http://example.com/Page.aspx",]
  
      def __init__(self):
          self.driver = webdriver.Firefox()
          dispatcher.connect(self.spider_closed, signals.spider_closed)
  
      def spider_closed(self, spider):
          self.driver.close()
  
      def parse(self, response):
  
          self.driver.get(response.url)
          item = MyItem()
  
          links = self.driver.find_elements_by_xpath("//input[@class='GetData']")
  
          for button in links:
              button.click()
              time.sleep(5)
  
              source = self.driver.page_source 
              sel = Selector(text=source) # create a Selector object
  
              item['url'] = self.driver.current_url
  
              print '\n\nURL\n', item['url'], '\n'
              yield item
  </code></pre>