title: Browserless web scraping of ajax page
body: <p>Have tried using Selenium after reading some tutorial for web scraping .. </p>
  
  <p>The aim is to web/screen scrape a page that loads the required data after an Ajax call when (this ajax call made after Initial page load).. </p>
  
  <p>The second aim is to run Selenium code in the background (not opening any browser) to allow loading the page (including the Ajax call) , retrieve the Final HTML and perform required processing locally .. </p>
  
  <p>the code till now is as follows (code tutorial from <a href="http://www.geekonweb.com/net/web-page-scraping-using-selenium-and-net/" rel="nofollow">http://www.geekonweb.com/net/web-page-scraping-using-selenium-and-net/</a>)</p>
  
  <pre><code>public ActionResult Index()
      {
          //--
          //Below path should contain IEDriverServer.exe
          var chrome = new ChromeDriver(@"file path");
          chrome.Url = @"&lt;url&gt;";
  
          chrome.Navigate();
  
          //extract the html
          //var retval = ie.ExecuteScript("return document.body.outerHTML");
  
          string result = chrome.PageSource;
  
  
          return View();
      }
  </code></pre>
  
  <p>currently have not been able to find a way to run Selenium Silently (without GUI). kindly assist if that can be done.</p>
  
  <p>Secondly kindly tell that how can Selenium be told to wait for the Ajax call to finish and then retrieve the data.</p>
  
  <p>regards,</p>