title: Javascript not loading for Selenium tests
body: <p>I'm trying to extract some data using Selenium for a pet project of mine.
  I've already loaded up a few pages successfully and got their data, however this one site stops loading everytime I test it.
  Things I have tried:</p>
  <ul>
  <li>Using geckodriver with Firefox both headless &amp; non-headless (headed??) versions</li>
  <li>Using chromedriver with Chrome both headless &amp; non-headless versions</li>
  <li>Checked that pip3 &amp; Selenium are all latest stable versions</li>
  <li>Opening Chrome with a user agent profile</li>
  <li>Opening Chrome with a random user agent profile (from random_user_agent library)</li>
  <li>Hardcoding waits for up to 30 seconds (time.sleep)</li>
  <li>Loading page in requests (in hindsight this was silly if I was looking for javascript - didn't work)</li>
  </ul>
  <p><a href="https://shop.coles.com.au/a/alexandria/product/nutella-spread-chocolate-hazelnut-2620684p" rel="nofollow noreferrer">The URL</a></p>
  <p>My theory is that they're blocking Selenium somehow, maybe <a href="https://stackoverflow.com/questions/33225947/can-a-website-detect-when-you-are-using-selenium-with-chromedriver">this</a>? But I have no way to test it. Issue does not persist when loading the page not using a Selenium browser instance (i.e. regular browser).
  My code below:</p>
  <pre><code>from selenium import webdriver
  
  # requirements to wait until specific part of page is open
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.common.by import By
  from selenium.common.exceptions import TimeoutException, NoSuchElementException
  from selenium.webdriver.chrome.options import Options
  
  options = Options()
  options.add_argument(&quot;--lang=en_US&quot;)
  options.add_experimental_option(&quot;excludeSwitches&quot;, [&quot;enable-automation&quot;])
  options.add_experimental_option(&quot;useAutomationExtension&quot;, False)
  options.add_argument(&quot;disable-infobars&quot;)
  
  browser = webdriver.Chrome(options=options)        
  delay = 5
  browser.get(&quot;https://shop.coles.com.au/a/alexandria/product/nutella-spread-chocolate-hazelnut-2620684p&quot;)
  # this is where the page is not loading &amp; therefore throwing ElementNotFound exception
  
  try:
      price_dollars = WebDriverWait(browser, delay).until(EC.presence_of_element_located((By.CLASS_NAME, &quot;price-dollars&quot;)))
      price_cents = browser.find_element_by_class_name(&quot;price-cents&quot;)
      
      # converts strings into floats with decimals (to one place only)
      fl_price_dollars = float(price_dollars.text)
      fl_price_cents = float(price_cents.text)
      fl_price_concat = fl_price_dollars + fl_price_cents*10**-2
      print(type(fl_price_concat)) # check this is a float type not string
      print(fl_price_concat)
  except TimeoutException:
      print(&quot;Timeout1&quot;)
      pass
  except NoSuchElementException:  # need to catch all exceptions &amp; pass to quit() or processes will continue to run
      print(&quot;Element not found&quot;)
      pass
  
  browser.quit()
  </code></pre>
  <p>Page source code that loads up when I open the page using Selenium browser instance:</p>
  <pre><code>
  &lt;!DOCTYPE html&gt;
  &lt;html lang=&quot;en&quot;&gt;
  &lt;head&gt;
      &lt;meta charset=&quot;UTF-8&quot;&gt;
      &lt;link rel=&quot;shortcut icon&quot; href=&quot;about:blank&quot;&gt;
  &lt;/head&gt;
  &lt;body&gt;
  &lt;script src=&quot;/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/j.js&quot;&gt;&lt;/script&gt;
  &lt;script src=&quot;/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/f.js&quot;&gt;&lt;/script&gt;
  &lt;script src=&quot;/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint/script/kpf.js?url=/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint&amp;token=c2e6cd9a-e76e-cd51-288d-f604aea52023&quot;&gt;&lt;/script&gt;
  &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  <p><strong>EDIT</strong> <a href="https://stackoverflow.com/questions/53039551/selenium-webdriver-modifying-navigator-webdriver-flag-to-prevent-selenium-detec">the following answer</a> worked for me in June 2020</p>