title: Python and Selenium: What's a good way to access links hidden in dropdown menus built by Javascript?
body: <p>I'm building a webcrawler in Python using Selenium. Below is the function that searches for links. It works fine, except for the part that's commented out. The idea is to hover over each link that was found on the page being examined, and if that hovering action reveals more links (like in a dropdown menu built with Javascript, for example), then add those links using a recursive call to getLinksFromHTML (the "if code != 1" part is just there to make sure there's only one recursive call).</p>
  
  <p>However, the recursive call doesn't pick up any new links when I test it on a page that has links inside JS dropdown menus (the page I'm looking at is <a href="http://wilmingtontaxesandaccounting.com" rel="nofollow">http://wilmingtontaxesandaccounting.com</a>). All visible links are picked up fine. </p>
  
  <p>What can I do so that Selenium sees those dropdown links? I need a general solution, i.e., no specific element IDs or anything else page-specific harcoded into the code. Thanks for reading!</p>
  
  <pre><code>def getLinksFromHTML(currUrl, code):
  
      ListlinkerHref = browser.find_elements_by_xpath("//*[@href]")
      links1 = []
      links2 = []
  
      for link in ListlinkerHref:
          url = link.get_attribute("href")
  
          #hov = ActionChains(browser).move_to_element(link)
          #hov.perform()
          #if code != 1:
              #links1 = self.getLinksFromHTML(currUrl, 1)
  
          if url not in links1:
              links2.append(url)
  
      return links1 + links2  
  </code></pre>