title: Add Proxy to Selenium & export dataframe to CSV
body: <p>I'm trying to make a scraper for capterra. I'm having issues getting blocked, so I think I need a proxy for my driver.get. Also, I am having trouble exporting a dataframe to a CSV. The first half of my code (not attached) is able to get all the links and store them in a list that I am trying to access with Selenium to get the information I want, but the second part is where I am having trouble.</p>
  <p>For an example, these are the types of links I am storing in the plinks list and that the driver is accessing:
  <a href="https://www.capterra.com/p/212448/Blackbaud-Altru/" rel="nofollow noreferrer">https://www.capterra.com/p/212448/Blackbaud-Altru/</a></p>
  <p><a href="https://www.capterra.com/p/80509/Volgistics-Volunteer-Management/" rel="nofollow noreferrer">https://www.capterra.com/p/80509/Volgistics-Volunteer-Management/</a></p>
  <p><a href="https://www.capterra.com/p/179048/One-Earth/" rel="nofollow noreferrer">https://www.capterra.com/p/179048/One-Earth/</a></p>
  <pre><code>for link in plinks:
      driver.get(link)
      #driver.implicitly_wait(20)
      companyProfile = bs(driver.page_source, 'html.parser')
      try:
          name = companyProfile.find(&quot;h1&quot;, class_=&quot;sm:nb-type-2xl nb-type-xl&quot;).text
      except AttributeError:
          name = &quot;couldn't find&quot;
      try:
          reviews = companyProfile.find(&quot;div&quot;, class_=&quot;nb-ml-3xs&quot;).text
      except AttributeError:
          reviews = &quot;couldn't find&quot;
      try:
          location = driver.find_element(By. XPATH, &quot;//*[starts-with(., 'Located in')]&quot;).text
      except NoSuchElementException:
          location = &quot;couldn't find&quot;
      try:
          url = driver.find_element(By. XPATH, &quot;//*[starts-with(., 'http')]&quot;).text
      except NoSuchElementException:
          url = &quot;couldn't find&quot;
      try:
          features = [x.get_text() for x in companyProfile.select('[id=&quot;LoadableProductFeaturesSection&quot;] li span')]
      except AttributeError:
          features = &quot;couldn't find&quot;
      companyInfo.append([name, reviews, location, url, features])
  
  companydf = pd.DataFrame(companyInfo, columns = [&quot;Name&quot;, &quot;Reviews&quot;, &quot;Location&quot;, &quot;URL&quot;, &quot;Features&quot;])
  companydf.to_csv(wmtest.csv, sep='\t')
  
  driver.close()
  </code></pre>
  <p>I am using Mozilla for the webdriver, and I am happy to change to Chrome if it works better, but is it possible to have the webdriver pick from a random set of proxies for each get request?</p>
  <p>Thanks!</p>