title: Ruby Selenium Webdriver - Need to wait / sleep while page redirects (refreshes)
body: <p>I'm using the Ruby <code>selenium-webdriver</code> gem to create a web-crawling/scraping script. The page that I'm scraping is loaded via AJAX and displays information for a certain account. If you select a second account number on a dropdown menu, the page redirects very briefly to another URL and back to the original URL, just with different information loaded via AJAX. I want to be able to scrape info for both the account numbers listed on the dropdown options. The problem is that Selenium performs the scrape faster than the page can redirect/reload on the dropdown click, so I don't end up getting the second account's information.</p>
  
  <pre><code>def crawl_page
    browser = Selenium::WebDriver.for :firefox
    browser.manage.timeouts.implicit_wait = 10 # seconds 
    browser.navigate.to 'http://www.foobar.com'
    account_dropdown = Selenium::WebDriver::Support::Select.new(browser.find_element(:id, 'account'))
    account_dropdown.options.each do |option|
      option.click
      wait = Selenium::WebDriver::Wait.new(:timeout =&gt; 10) # seconds
      # this wait is not working because option is selected before redirect/refresh:
      wait.until { option.selected? }
      html = browser.page_source
      scrape_page(html)
    end
    browser.quit  
  end
  </code></pre>
  
  <p>I've tried putting a <code>sleep(3)</code> on the line after the click, but get the following error message:</p>
  
  <pre><code>[remote server] resource://fxdriver/modules/web_element_cache.js:8180:in `fxdriver.cache.getElementAt': Element not found in the cache - perhaps the page has changed since it was looked up (Selenium::WebDriver::Error::StaleElementReferenceError)
  </code></pre>
  
  <p>I've also tried using Selenium's explicit <code>wait</code> code, but the ids of the elements appear to dynamically change on the updated page so something like:</p>
  
  <pre><code>wait = Selenium::WebDriver::Wait.new(:timeout =&gt; 10) # seconds
  wait.until { browser.find_element(:id, 'titlexyz').displayed? }
  </code></pre>
  
  <p>results in an error message that says it's timed out and can't find the element:</p>
  
  <pre><code>~lib/selenium/webdriver/common/wait.rb:57:in `until': timed out after 10 seconds (Unable to locate element: {"method":"id","selector":"titlexyz"}) (Selenium::WebDriver::Error::TimeOutError)
  </code></pre>
  
  <p>Is there some way to get it to sleep or wait without having to look for a specific element on the page?</p>