title: Mechanize for Python and authorization without first getting a 401 error
body: <p>I'm trying to use Mechanize to automate interactions with a very picky legacy system. In particular, after the first login page the authorization must be sent with every request it knocks you out of the system. Unfortunately, Mechanize seems content on only sending the authorization after first getting a 401 Unauthorized error. Is there any way to have it send authorization every time?</p>
  
  <p>Here's some sample code:</p>
  
  <pre><code>br.add_password("http://example.com/securepage", "USERNAME", "PASSWORD", "/MYREALM")
  br.follow_link(link_to_secure_page) # where the url is the previous URL
  </code></pre>
  
  <p>Here's the response I get from debugging Mechanize:</p>
  
  <pre><code>send: 'GET /securepage HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: example.com\r\nReferer: http://example.com/home\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n'
  reply: 'HTTP/1.1 401 Unauthorized\r\n'
  header: Server: Tandy1000Web
  header: Date: Thu, 08 Dec 2011 03:08:04 GMT
  header: Connection: close
  header: Expires: Tue, 01 Jan 1980 06:00:00 GMT
  header: Content-Type: text/html; charset=US-ASCII
  header: Content-Length: 210
  header: WWW-Authenticate: Basic realm="/MYREALM"
  header: Cache-control: no-cache
  send: 'GET /securepage HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: example.com\r\nReferer: http://example.com/home\r\nConnection: close\r\nAuthorization: Basic VVNFUk5BTUU6UEFTU1dPUkQ=\r\nUser-Agent: Python-urllib/2.7\r\n\r\n'
  reply: 'HTTP/1.1 200 OK\r\n'
  header: Server: Tandy1000Web
  header: Date: Thu, 08 Dec 2011 03:08:07 GMT
  header: Connection: close
  header: Last-Modified: Thu, 08 Dec 2011 03:08:06 GMT
  header: Expires: Tue, 01 Jan 1980 06:00:00 GMT
  header: Content-Type: text/html; charset=UTF-8
  header: Content-Length: 33333
  header: Cache-control: no-cache
  </code></pre>
  
  <p>The problem is that contrary to what should happen in modern web application with a <code>GET</code> request, by hitting the 401 error first I get the wrong page. I've confirmed with CURL and urllib2 that if I hit the URL directly by passing in the auth header on the first request I get the correct page.</p>
  
  <p>Any hints on how to tell mechanize to always send the auth headers and avoid the first 401 error? This needs to be fixed on the client side. I can't modify the server.</p>