title: Simulating clicking on a javascript link in python
body: <p>I am trying to collate reviews of restaurants. Urllib2 works fine for the initial page of reviews, but there is then a link to load the next increment of comments which is a javascript  link. An example page is <a href="http://www.just-eat.co.uk/restaurants-chathamgrill-me4/reviews" rel="nofollow noreferrer">here</a>, and the code for the link "Next 25" is:</p>
  
  <pre><code>&lt;a href="javascript:__doPostBack('ctl00$ContentPlaceHolder1$RestRatings$Next','')" class="red" id="ctl00_ContentPlaceHolder1_RestRatings_Next"&gt;NEXT 25&amp;gt;&amp;gt; &lt;/a&gt;
  </code></pre>
  
  <p>I have looked at all the previous answers (<a href="https://stackoverflow.com/questions/11018796/clicking-on-a-javascript-link-on-firefox-with-selenium">e.g.</a>), and I have to say I'm none the wiser. Looking at the console in Firebug doesn't offer up a handy link. Could you suggest the best (easiest) way to achieve this?</p>
  
  <p>Edit:
  With thanks to Seleniumnewbie this code will print out all the comments from the reviews.:</p>
  
  <pre><code>from selenium import webdriver
  from BeautifulSoup import BeautifulSoup
  import re
  
  driver = webdriver.Firefox()
  
  def getURLinfo(url):
  
      driver.get(url)
      html = driver.page_source
      next25 = "ctl00_ContentPlaceHolder1_RestRatings_Next"
      soup = BeautifulSoup(html)
  
      while soup.find(id=re.compile(next25)):            
          driver.find_element_by_id(next25).click()
          html = html + driver.page_source
          soup = BeautifulSoup(driver.page_source)
  
      soup = BeautifulSoup(html)
      comment = soup.findAll(id=re.compile("divComment"))
  
      for entry in comment:
          print entry.div.contents #for comments
  
      driver.close()
  </code></pre>