title: Web scraping with dryscrape: error selecting radio button CSS
body: <p>I am trying to get data from a table that is updated with javascript inside a web page (<a href="http://www.madrid.org/wpad_pub/run/j/MostrarFichaCentro.icm?cdCentro=28063799" rel="nofollow noreferrer">http://www.madrid.org/wpad_pub/run/j/MostrarFichaCentro.icm?cdCentro=28063799</a>), and I am using dryscrape. I have a code that works well with tables that are generated by default when the page loads. But I need to update one of them by clicking on a radio button (radio button labeled "Primary" in the second table).</p>
  
  <p>I changed my code, and it looks like this:</p>
  
  <pre><code>from bs4 import BeautifulSoup
  import pandas as pd
  import dryscrape
  
  render = dryscrape.Session()
  render.visit("http://www.madrid.org/wpad_pub/run/j/MostrarFichaCentro.icm?cdCentro=28063799")
  radiob = render.at_css('#nivEd12\.grafica3')
  radiob.click()
  source = render.body()
  school_card = BeautifulSoup(source, "lxml")
  school_tables = school_card.findAll('table', class_="tablaGraficaDatos")
  table = list(school_tables)[1]
  pd.read_html(table.prettify())
  </code></pre>
  
  <p>But I get the following error:</p>
  
  <pre><code>InvalidResponseError: {"class":"ClickFailed","message":"Failed to find position for element /html/body/div[@id='contenedor']/div[@id='solapas']/div[10]/table/tbody/tr[1]/td[1]/div[@id='solapaspanel1']/div[@id='cuerpoL']/div/div[@id='capaSelGrafica']/div[@id='display.grafica3']/table/tbody/tr[2]/td[2]/input[@id='nivEd12.grafica3'] because it is not visible"}
  </code></pre>
  
  <p>I have tried with also with xpath:</p>
  
  <pre><code>radiob = render.at_xpath('//*[(@id = "nivEd12.grafica3")]')
  </code></pre>
  
  <p>But I get the same error.</p>
  
  <p>I have used Selector Gadget to get CSS and XPath. I imagine there is some error in the path of the radio button, but I do not know how to fix it. Any idea?</p>
  
  <p>Thanks in advance.</p>
  
  <hr>
  
  <p>UPDATE</p>
  
  <p>@CtheSky has given me a solution that works fine with singles urls. But when I try to loop to multiple urls, I get an error. This is the script.</p>
  
  <pre><code>schools_urls2 = ['http://www.madrid.org/wpad_pub/run/j/MostrarFichaCentro.icm?cdCentro=28077865',
   'http://www.madrid.org/wpad_pub/run/j/MostrarFichaCentro.icm?cdCentro=28063751',
   'http://www.madrid.org/wpad_pub/run/j/MostrarFichaCentro.icm?cdCentro=28004989',
   'http://www.madrid.org/wpad_pub/run/j/MostrarFichaCentro.icm?cdCentro=28004990']
  
  school_tables_collection = {}
  school_name_collection = []
  
  render = dryscrape.Session()
  for z, school in enumerate(schools_urls[:5]):
      render.visit(school)
      render.driver.exec_script('document.getElementById("nivEd12.grafica3").click();')
      source = render.body()
      school_card = BeautifulSoup(source, "lxml")
      school_tables = school_card.findAll('table', class_="tablaGraficaDatos")
      school_name = school_card.find(style="text-transform:uppercase").next.next
      for i, table in list(enumerate(school_tables)):
          if i &lt;= 1:
              school_tables_collection[school_name + "_" + str(i)] = \
              pd.read_html(table.prettify())
              school_name_collection.append(school_name)
      print "Tables of school %s extracted" % schools_urls[z]
  </code></pre>
  
  <p>Any idea about what I'm doing wrong?</p>
  
  <hr>
  
  <p>SOLUTION</p>
  
  <p>I've finally managed to fix it. It was a silly mistake on my part: the first url I called did not have the button element I was looking for, so it returned error. I've included a <code>try</code> and <code>except</code> in the loop and now it works.</p>
  
  <p>Thank you very much for your help @CtheSky</p>