title: Selenium headless: How to bypass Cloudflare detection using Selenium
body: <p>Hoping an expert can help me with a Selenium/Cloudflare mystery. I can get a website to load in normal (non-headless) Selenium, but no matter what I try, I can't get it to load in headless.</p>
  <p>I have followed the suggestions from the StackOverflow posts like <a href="https://stackoverflow.com/questions/56528631/is-there-a-version-of-selenium-webdriver-that-is-not-detectable">Is there a version of Selenium WebDriver that is not detectable?</a>. I've also looked at all the properties of <code>window</code> and <code>window.navigator</code> objects and fixed all the diffs between headless and non-headless, but somehow headless is still being detected. At this point I am extremely curious how Cloudflare could possibly figure out the difference. Thank you for the time!</p>
  <p><strong>List of the things I have tried:</strong></p>
  <ul>
  <li>User-agent</li>
  <li>Replace <code>cdc_</code> with another string in chromedriver</li>
  <li><code>options.add_experimental_option(&quot;excludeSwitches&quot;, [&quot;enable-automation&quot;])</code></li>
  <li><code>options.add_experimental_option('useAutomationExtension', False)</code></li>
  <li><code>options.add_argument('--disable-blink-features=AutomationControlled')</code> (this was necessary to get website to load in non-headless)</li>
  <li>Set <code>navigator.webdriver = undefined</code></li>
  <li>Set <code>navigator.plugins</code>, <code>navigator.languages</code>, and <code>navigator.mimeTypes</code></li>
  <li>Set <code>window.ScreenY</code>, <code>window.screenTop</code>, <code>window.outerWidth</code>, <code>window.outerHeight</code> to be nonzero</li>
  <li>Set <code>window.chrome</code> and <code>window.navigator.chrome</code></li>
  <li>Set width and height of images to be nonzero</li>
  <li>Set WebGL parameters</li>
  <li>Fix <code>Modernizr</code></li>
  </ul>
  <p><strong>Replicating the experiment</strong></p>
  <p>In order to get the website to load in normal (non-headless) Selenium, you have to follow a <code>_blank</code> link from another website (so that the target website opens in another tab). To replicate the experiment, first create an html file with the content <code>&lt;a href=&quot;https://poocoin.app&quot; target=&quot;_blank&quot;&gt;link&lt;/a&gt;</code>, and then paste the path to this html file in the following code.</p>
  <p>The version below (non-headless) runs fine and loads the website, but if you set <code>options.headless = True</code>, it will get stuck on Cloudflare.</p>
  <pre><code>from selenium import webdriver
  import time
  
  # Replace this with the path to your html file
  FULL_PATH_TO_HTML_FILE = 'file:///Users/simplepineapple/html/url_page.html'
  
  def visit_website(browser):
      browser.get(FULL_PATH_TO_HTML_FILE)
      time.sleep(3)
  
      links = browser.find_elements_by_xpath(&quot;//a[@href]&quot;)
      links[0].click()
      time.sleep(10)
  
      # Switch webdriver focus to new tab so that we can extract html
      tab_names = browser.window_handles
      if len(tab_names) &gt; 1:
          browser.switch_to.window(tab_names[1])
  
      time.sleep(1)
      html = browser.page_source
      print(html)
      print()
      print()
  
      if 'Charts' in html:
          print('Success')
      else:
          print('Fail')
  
      time.sleep(10)
  
  
  options = webdriver.ChromeOptions()
  # If options.headless = True, the website will not load
  options.headless = False
  options.add_argument(&quot;--window-size=1920,1080&quot;)
  options.add_experimental_option(&quot;excludeSwitches&quot;, [&quot;enable-automation&quot;])
  options.add_experimental_option('useAutomationExtension', False)
  options.add_argument('--disable-blink-features=AutomationControlled')
  options.add_argument('user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36')
  
  browser = webdriver.Chrome(options = options)
  
  browser.execute_cdp_cmd('Page.addScriptToEvaluateOnNewDocument', {
      &quot;source&quot;: '''
      Object.defineProperty(navigator, 'webdriver', {
          get: () =&gt; undefined
      });
      Object.defineProperty(navigator, 'plugins', {
              get: function() { return {&quot;0&quot;:{&quot;0&quot;:{}},&quot;1&quot;:{&quot;0&quot;:{}},&quot;2&quot;:{&quot;0&quot;:{},&quot;1&quot;:{}}}; }
      });
      Object.defineProperty(navigator, 'languages', {
          get: () =&gt; [&quot;en-US&quot;, &quot;en&quot;]
      });
      Object.defineProperty(navigator, 'mimeTypes', {
          get: function() { return {&quot;0&quot;:{},&quot;1&quot;:{},&quot;2&quot;:{},&quot;3&quot;:{}}; }
      });
  
      window.screenY=23;
      window.screenTop=23;
      window.outerWidth=1337;
      window.outerHeight=825;
      window.chrome =
      {
        app: {
          isInstalled: false,
        },
        webstore: {
          onInstallStageChanged: {},
          onDownloadProgress: {},
        },
        runtime: {
          PlatformOs: {
            MAC: 'mac',
            WIN: 'win',
            ANDROID: 'android',
            CROS: 'cros',
            LINUX: 'linux',
            OPENBSD: 'openbsd',
          },
          PlatformArch: {
            ARM: 'arm',
            X86_32: 'x86-32',
            X86_64: 'x86-64',
          },
          PlatformNaclArch: {
            ARM: 'arm',
            X86_32: 'x86-32',
            X86_64: 'x86-64',
          },
          RequestUpdateCheckStatus: {
            THROTTLED: 'throttled',
            NO_UPDATE: 'no_update',
            UPDATE_AVAILABLE: 'update_available',
          },
          OnInstalledReason: {
            INSTALL: 'install',
            UPDATE: 'update',
            CHROME_UPDATE: 'chrome_update',
            SHARED_MODULE_UPDATE: 'shared_module_update',
          },
          OnRestartRequiredReason: {
            APP_UPDATE: 'app_update',
            OS_UPDATE: 'os_update',
            PERIODIC: 'periodic',
          },
        },
      };
      window.navigator.chrome =
      {
        app: {
          isInstalled: false,
        },
        webstore: {
          onInstallStageChanged: {},
          onDownloadProgress: {},
        },
        runtime: {
          PlatformOs: {
            MAC: 'mac',
            WIN: 'win',
            ANDROID: 'android',
            CROS: 'cros',
            LINUX: 'linux',
            OPENBSD: 'openbsd',
          },
          PlatformArch: {
            ARM: 'arm',
            X86_32: 'x86-32',
            X86_64: 'x86-64',
          },
          PlatformNaclArch: {
            ARM: 'arm',
            X86_32: 'x86-32',
            X86_64: 'x86-64',
          },
          RequestUpdateCheckStatus: {
            THROTTLED: 'throttled',
            NO_UPDATE: 'no_update',
            UPDATE_AVAILABLE: 'update_available',
          },
          OnInstalledReason: {
            INSTALL: 'install',
            UPDATE: 'update',
            CHROME_UPDATE: 'chrome_update',
            SHARED_MODULE_UPDATE: 'shared_module_update',
          },
          OnRestartRequiredReason: {
            APP_UPDATE: 'app_update',
            OS_UPDATE: 'os_update',
            PERIODIC: 'periodic',
          },
        },
      };
      ['height', 'width'].forEach(property =&gt; {
          const imageDescriptor = Object.getOwnPropertyDescriptor(HTMLImageElement.prototype, property);
  
          // redefine the property with a patched descriptor
          Object.defineProperty(HTMLImageElement.prototype, property, {
              ...imageDescriptor,
              get: function() {
                  // return an arbitrary non-zero dimension if the image failed to load
              if (this.complete &amp;&amp; this.naturalHeight == 0) {
                  return 20;
              }
                  return imageDescriptor.get.apply(this);
              },
          });
      });
  
      const getParameter = WebGLRenderingContext.getParameter;
      WebGLRenderingContext.prototype.getParameter = function(parameter) {
          if (parameter === 37445) {
              return 'Intel Open Source Technology Center';
          }
          if (parameter === 37446) {
              return 'Mesa DRI Intel(R) Ivybridge Mobile ';
          }
  
          return getParameter(parameter);
      };
  
      const elementDescriptor = Object.getOwnPropertyDescriptor(HTMLElement.prototype, 'offsetHeight');
  
      Object.defineProperty(HTMLDivElement.prototype, 'offsetHeight', {
          ...elementDescriptor,
          get: function() {
              if (this.id === 'modernizr') {
              return 1;
              }
              return elementDescriptor.get.apply(this);
          },
      });
      '''
  })
  
  visit_website(browser)
  
  browser.quit()
  </code></pre>