title: Web scraping using VB.NET
body: <p>I have a url:</p>
  
  <pre><code>test.com/Search/NumberSearch.aspx
  </code></pre>
  
  <p>On the page there are a number of controls, one of them is textbox. When the user enters a six digit (approximately) number into the textbox and hits enter, the page goes to another page:</p>
  
  <pre><code>test.com/Data/DetailsPage.aspx?mynum=123456
  </code></pre>
  
  <p>on that page there are a number of textboxes and other controls from which I need to scrape the data in addition to a number of links that I need to caputure in my code.</p>
  
  <p>I have tried using VB.NET WebRequest:</p>
  
  <pre><code>Dim wreq As WebRequest = WebRequest.Create("test.com/Data/DetailsPage.aspx?mynum=" &amp; num)
  Dim wresp As HttpWebResponse = CType(wreq.GetResponse(), HttpWebResponse)  
  Dim dStream As Stream = wresp.GetResponseStream()
  Dim rdr As New StreamReader(dStream)
  Dim respStr As String = rdr.ReadToEnd()
  </code></pre>
  
  <p>As a result my respStr contains a string with html code but that code is for</p>
  
  <pre><code> test.com/Search/NumberSearch.aspx
  </code></pre>
  
  <p>not for the resulting     </p>
  
  <pre><code>test.com/Data/DetailsPage.aspx?mynum=123456
  </code></pre>
  
  <p>page with details. </p>
  
  <p>My goal is to get the details page html programmatically. </p>
  
  <p>I also tried using</p>
  
  <pre><code>WebClient.DownloadString
  </code></pre>
  
  <p>but gotten the same result. Can anyone help?</p>