title: Programmatically login to web page HTTPS (C#)
body: <p>I'm trying to login programmatically to a web page and I'm having any joy. I had a look at fiddler to see what the browser is doing but I'm struggling to replicate it in code.
  Here is the code so far.</p>
  
  <pre><code>private void VisitUrl()
  {
      string pageVisited = string.Empty;
      CookieContainer cookies = new CookieContainer();
  
      //Go to login page so that we get a cookie
      HttpWebRequest loginRequest = (HttpWebRequest)HttpWebRequest.Create("http://www.apage.com/login.asp");
  
      loginRequest.UseDefaultCredentials = true;
      loginRequest.UserAgent = Constants.usrAgentIE8;
      loginRequest.CookieContainer = cookies;
      loginRequest.AllowAutoRedirect = false;
  
      HttpWebResponse loginResponse = (HttpWebResponse)loginRequest.GetResponse();
  
      string guid = Regex.Match(loginResponse.Headers["Set-Cookie"], Constants.guidRegex).Value;
      guid = guid.Replace("%2D", "-");
  
      //Now that we have the cookie we can do a proper login
      HttpWebRequest doLogin = (HttpWebRequest)HttpWebRequest.Create("https://secure.apage.com/express/login.asp");
  
      //doLogin.UseDefaultCredentials = true;
      doLogin.UserAgent = Constants.usrAgentIE8;
      doLogin.Method = Constants.verbPOST;
      doLogin.ContentType = Constants.contTypeURLenc;
      doLogin.AllowAutoRedirect = false;
  
      using (StreamWriter sw = new StreamWriter(doLogin.GetRequestStream()))
      {
          sw.Write("usr=user&amp;pass=" + Constants.pass + "&amp;mc=1&amp;guid=" + guid + "&amp;dbnum=0&amp;ic=1&amp;task=dologin&amp;app=library&amp;x=32&amp;y=10");
  
      }
  
      doLogin.CookieContainer = cookies;
      doLogin.Referer = "http://www.apage.com/login.asp";
  
      HttpWebResponse Response = (HttpWebResponse)doLogin.GetResponse();
  
      string resphead = Response.Headers["Set-Cookie"];
  
      string resp = Response.StatusCode.ToString();
  
      using (StreamReader sr = new StreamReader(Response.GetResponseStream()))
      {
          string textresponse = sr.ReadToEnd();
      }
  
      HttpWebRequest Request = (HttpWebRequest)HttpWebRequest.Create("http://www.apage.com/toc.aspx?id=40729");
      Request.UserAgent = Constants.usrAgentIE8;
      Request.UseDefaultCredentials = true;
      Request.CookieContainer = cookies;
  
      using (HttpWebResponse urlResponse = (HttpWebResponse)Request.GetResponse())
      {
          using (StreamReader reader = new StreamReader(urlResponse.GetResponseStream()))
          {
              pageVisited = reader.ReadToEnd();
          }
      }
  }
  </code></pre>
  
  <p>The problem I have is that when I try the third request, <a href="http://www.apage.com/toc.aspx?id=40729" rel="nofollow noreferrer">http://www.apage.com/toc.aspx?id=40729</a>, it is just missing a cookie, and it seems me back to the login page. This is a successful login with a browser from fiddler:</p>
  
  <pre><code> 1  200 HTTP    CONNECT secure.apage.com:443    0           iexplore:5908           
   2  200 HTTPS   secure.apage.com    /express/login.asp  3,253   private     text/html   iexplore:5908           
   3  200 HTTPS   secure.apage.com    /express/zarketing.css  13,340      text/css    iexplore:5908           
   4  200 HTTPS   secure.apage.com    /express/images/_loadingtext.gif    798     image/gif   iexplore:5908           
   5  200 HTTP    CONNECT secure.apage.com:443    0           iexplore:5908           
   6  200 HTTP    CONNECT secure.apage.com:443    0           iexplore:5908           
   7  200 HTTP    CONNECT secure.apage.com:443    0           iexplore:5908           
   8  200 HTTP    CONNECT secure.apage.com:443    0           iexplore:5908           
   9  200 HTTP    CONNECT secure.apage.com:443    0           iexplore:5908           
   10 200 HTTPS   secure.apage.com    /express/images/_loadingbox1.gif    66      image/gif   iexplore:5908           
   11 200 HTTPS   secure.apage.com    /express/images/_loadingbox5.gif    66      image/gif   iexplore:5908           
   12 200 HTTPS   secure.apage.com    /express/images/_loadingbox4.gif    100     image/gif   iexplore:5908           
   13 200 HTTPS   secure.apage.com    /express/images/_loadingbox2.gif    66      image/gif   iexplore:5908           
   14 200 HTTPS   secure.apage.com    /express/images/_loadingbox3.gif    66      image/gif   iexplore:5908           
   15 302 HTTP    www.apage.com   /authenticate.asp?ticket=2ECCC1686F49BE35E14556E76EFCBF3E478157F89B826A03C421883BE61BB634D8DDDB25F8943414252CE46C27B1FAE837B802A7BD1EE9723ECDCB99B7D3D1DB60FAB3042D0D632BF1C41BA622C2546E&amp;mc=1  142 max-age=0, no-cache, no-store  Expires: Fri, 10 Jun 2011 09:04:21 GMT   text/html   iexplore:5908           
   16 200 HTTP    www.apage.com   /toc.aspx?id=40729  86,179  max-age=0, no-cache, no-store  Expires: Fri, 10 Jun 2011 09:04:22 GMT   text/html; charset=utf-8    iexplore:5908           
  </code></pre>
  
  <p>Request 15 is a redirect and this request already has the missing cookie, but I don't know how this cookie is created. It does not seem to be present on response from my second request, <a href="https://secure.apage.com/express/login.asp" rel="nofollow noreferrer">https://secure.apage.com/express/login.asp</a> , I've looked while debugging.</p>
  
  <p>Do I need to make requests for all the gifs?</p>