title: HTML in selenium firefox webdriver not loading
body: <p>I am trying to scrape a website using <code>Selenium Firefox</code> (headless) driver in <code>python</code>.</p>
  
  <p>I read all the anchors in the webpage and go through them all one by one. But I want for the browser to wait for the <code>Ajax</code> calls on the page to be over before moving to another page.</p>
  
  <p>My code is the following:</p>
  
  <pre><code>import time 
  from selenium import webdriver
  from selenium.webdriver.firefox.options import Options
  from selenium.webdriver.support.ui import WebDriverWait
  
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
  
  caps = DesiredCapabilities().FIREFOX
  caps["pageLoadStrategy"] = "eager"  #  complete
  
  options = Options()
  options.add_argument("--headless")
  
  url = "http://localhost:3000/"
  
  # Using Selenium's webdriver to open the page
  driver = webdriver.Firefox(desired_capabilities=caps,firefox_options=options)
  driver.get(url)
  urls = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.TAG_NAME, "a")))
  
  links = []
  
  for url in urls:
      links.append(url.get_attribute("href"))
  
  for link in links:
      print 'navigating to: ' + link
      driver.get(link)
      body = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.TAG_NAME, "p")))
      driver.execute_script("window.scrollTo(0,1000);")
      print(body)    
      driver.back()
  
  driver.quit()
  </code></pre>
  
  <p>the line <code>print(body)</code> was added for testing purposes. and it returned uncomprehensible text , instead of the actual HTML of the page. Heres a part of the printed text :</p>
  
  <pre><code>[&lt;selenium.webdriver.firefox.webelement.FirefoxWebElement (session="fb183e8b-ce36-47e7-a03e-d3aeea376304", element="e7dfa6b2-1ddf-438d-b562-1e2ac8416e07")&gt;, &lt;selenium.webdriver.firefox.webelement.FirefoxWebElement (session="fb183e8b-ce36-47e7-a03e-d3aeea376304", element="6fe1ffb0-17a8-4b64-9166-691478a0bbd4")&gt;, &lt;selenium.webdriver.firefox.webelement.FirefoxWebElement (session="fb183e8b-ce36-47e7-a03e-d3aeea376304", element="1f510a00-a587-4ae8-9ecf-dd4c90081a5a")&gt;, &lt;selenium.webdriver.firefox.webelement.FirefoxWebElement (session="fb183e8b-ce36-47e7-a03e-d3aeea376304", element="c1bfb1cd-5ccf-42b6-ad4c-c1a70486cc98")&gt;, &lt;selenium.webdriver.firefox.webelement.FirefoxWebElement (session="fb183e8b-ce36-47e7-a03e-d3aeea376304", element="be44db09-3948-48f1-8505-937db509a157")&gt;, &lt;selenium.webdriver.firefox.webelement.FirefoxWebElement (session="fb183e8b-ce36-47e7-a03e-d3aeea376304", element="68f3c9f2-80b0-493e-a47f-ad69caceaa06")&gt;, 
  </code></pre>
  
  <p>What is causing this?</p>
  
  <p>Everything (content related) in the pages i'm scraping is static.</p>