title: Regex and POST in same connection
body: <p>please bear with me, i'm brand new to Python!</p>
  
  <p>I'm trying to login to a website which uses PHP. The form contains two hidden fields, the value of one and the name of another are generated on page load.</p>
  
  <p>My code below successfuly accesses the page and using regex manages to return the values - great!</p>
  
  <p>The problem I am having is that I then generate my querystring that will be used for the POST (this contains the two values obtained earlier) and opens the url again. This generates brand new tokens/values and my originals are of no use.</p>
  
  <p>Can someone shed some light on how I can connect to a site, use regex to get the values and then POST all in the same connection.</p>
  
  <p>I hope i've made myself clear, if not please let me know.</p>
  
  <p>Thanks in advance for your help.</p>
  
  <pre><code>import urllib2,urllib,re,cookielib
  
  url='http://www.example.com/index.php'
  
  req = urllib2.Request(url)
  req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3 Gecko/2008092417 Firefox/3.0.3')
  response = urllib2.urlopen(req)
  link=response.read()
  response.close()
  token1=re.compile('&lt;input type="hidden" name="return" value="(.+?)" /&gt;').findall(link)
  token2=re.compile('&lt;input type="hidden" name="(.+?)" value="1" /&gt;').findall(link)
  print token1[0]
  print token2[0]
  
  username = 'username'
  password = 'password'
  
  cj = cookielib.CookieJar()
  opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
  login_data = urllib.urlencode({'username' : username, 'password' : password, 'return' : token1[0], token2[0] : '1', 'Submit' : 'Log in', 'option' : 'com_users', 'task' : 'user.login'})
  opener.open('http://www.example.com/index.php', login_data)
  resp = opener.open('http://www.example.com/index.php')
  </code></pre>
  
  <p>FORM:</p>
  
  <pre><code>&lt;form action="/index.php/welcome2" method="post" id="login-form" &gt;
       &lt;fieldset class="userdata"&gt;
            &lt;p id="form-login-username"&gt;
                 &lt;label for="modlgn-username"&gt;User Name&lt;/label&gt;
                 &lt;input id="modlgn-username" type="text" name="username" class="inputbox"  size="18" /&gt;
            &lt;/p&gt;
            &lt;p id="form-login-password"&gt;
                 &lt;label for="modlgn-passwd"&gt;Password&lt;/label&gt;
                 &lt;input id="modlgn-passwd" type="password" name="password" class="inputbox" size="18"  /&gt;
            &lt;/p&gt;
            &lt;p id="form-login-remember"&gt;
                 &lt;label for="modlgn-remember"&gt;Remember Me&lt;/label&gt;
                 &lt;input id="modlgn-remember" type="checkbox" name="remember" class="inputbox" value="yes"/&gt;
           &lt;/p&gt;
           &lt;input type="submit" name="Submit" class="button" value="Log in" /&gt;
           &lt;input type="hidden" name="option" value="com_users" /&gt;
           &lt;input type="hidden" name="task" value="user.login" /&gt;
           &lt;input type="hidden" name="return" value="aW5kZXgucGhwP0l0ZW1pZD0xMjc=" /&gt;
           &lt;input type="hidden" name="c813c34837e4e48e8e3268c0a42912a2" value="1" /&gt;
      &lt;/fieldset&gt;
  &lt;ul&gt;
  &lt;li&gt;
  &lt;a href="/index.php/my-account/my-details?view=reset"&gt;
  Forgot your password?&lt;/a&gt;
  &lt;/li&gt;
  &lt;li&gt;
  &lt;a href="/index.php/my-account/my-details?view=remind"&gt;
  Forgot your username?&lt;/a&gt;
  &lt;/li&gt;
  &lt;li&gt;
  &lt;a href="/index.php/register"&gt;
  Create an account&lt;/a&gt;
  &lt;/li&gt;
  &lt;/ul&gt;
  &lt;/form&gt;
  </code></pre>