title: How to stay logged into a website using a Python script (requests library)?
body: <p>I am attempting to use the Python <a href="http://requests.readthedocs.org/en/latest/" rel="nofollow">requests</a> library to login to a website called surfline.com, then <code>get</code> a webpage once logged in (presumably within a persisting <a href="http://docs.python-requests.org:8000/en/latest/user/advanced/" rel="nofollow">session</a>).</p>
  
  <p>The login form that surfline.com uses throughout its pages uses <code>onclick()</code> to map to a JS function called <code>verifyLogin()</code>, which in turn posts to a <code>.cfm</code> file. On <code>success</code>, it refreshes the page, and the user is now kept logged in so long as the cookies persist.</p>
  
  <p>I am using the <a href="http://requests.readthedocs.org/en/latest/" rel="nofollow">requests</a> library for the first time, and am unsure how to:</p>
  
  <ol>
  <li>login successfully, </li>
  <li>stay logged-in throughout the session, </li>
  <li>then print out the homepage as a logged-in user to the terminal.</li>
  </ol>
  
  <p>Here is the login form (HTML):</p>
  
  <pre><code>&lt;form id="loginForm"&gt;
      &lt;label for="name"&gt;Email Address:&lt;/label&gt;&lt;br&gt;
      &lt;input type="text" name="username" id="username"&gt;
      &lt;label for="mail"&gt;Password:&lt;/label&gt;&lt;br&gt;
      &lt;input type="password" name="password" id="password"&gt;
      &lt;input type="hidden" name="top_login" id="top_login" value="true"&gt;
      &lt;button class="surfline-button blue1" type="button" onclick="verifyLogin();"&gt;Log In&lt;/button&gt;
      &lt;button class="surfline-button grey1" type="button" onclick="jQuery('#dialog-login').dialog('close');"&gt;Cancel&lt;/button&gt;
      &lt;div id="forgot"&gt;Forgot Password? &lt;a href="/myaccount/?action=forgot_password"&gt;Click Here&lt;/a&gt;&lt;/div&gt;
      &lt;div class="clear"&gt;&lt;/div&gt;
      &lt;div&gt;&lt;input style="float:left; width:18px; height:18px; margin-right:6px; border:none" type="checkbox" name="rememberMe" id="rememberMe" value="true" checked="checked"&gt;&lt;div id="remember-me-text" style="float:left; width:200px; text-align:left; margin-top:1px;"&gt;Remember me on this computer?&lt;/div&gt;&lt;/div&gt;
      &lt;div class="clear"&gt;&lt;/div&gt;
      &lt;p&gt;Not a Premium member?  &lt;a href="https://www.surfline.com/subscribe_vindicia/index.cfm?mkt=login&amp;amp;slintcid=LOGIN&amp;amp;slcmpname=LOGIN-MODAL"&gt;TRY PREMIUM FREE NOW&lt;/a&gt;&lt;/p&gt;
  &lt;/form&gt;
  </code></pre>
  
  <p>Here is the <code>verifyLogin()</code> function in the <a href="http://www.surfline.com/global_includes/scripts/05222013_slmenu.js" rel="nofollow">/05222013_slmenu.js</a> file:</p>
  
  <pre><code>function verifyLogin(){  
      //var username = jQuery("#username").val();
      //var password = jQuery("#password").val();
      var usernameVal = jQuery("#username").val();
      var passwordVal = jQuery("#password").val();
      var rememberMeVal = jQuery("#rememberMe").val();    
      var top_loginVal = jQuery("#top_login").val(); 
  
      if(usernameVal.length === 0 || passwordVal.length === 0 ){
          jQuery("#login-note").addClass("warning");
          if(usernameVal.length === 0){ jQuery("#username").addClass('warning'); jQuery("#login-note").html("Email Field is Blank"); }else{ jQuery("#username").removeClass('warning');}
          if(passwordVal.length === 0){ jQuery("#password").addClass('warning'); jQuery("#login-note").html("Password Field is Blank"); }else{ jQuery("#password").removeClass('warning'); }
          if(usernameVal.length === 0 &amp;&amp; passwordVal.length === 0){jQuery("#login-note").html("Email and Password Fields are Blank"); }
      }else{  
          jQuery("#inner-dialog").fadeOut("slow",function(){
              var htmlData = "&lt;center&gt;&lt;div style='padding-top:60px;'&gt;&lt;h1&gt;Verifying Login&lt;/h1&gt;&lt;img src='/global_includes/images/ajax-loader-snake-295284.gif' style='margin-top:24px;'&gt;&lt;/div&gt;&lt;/center&gt;";                                                
              jQuery("#verifying").html(htmlData).fadeIn("slow", function(){ 
                  //var loginData = jQuery('#loginForm').serialize();
  
                  var loginData = 'username=' + escape(usernameVal) + '&amp;password=' + escape(passwordVal) + '&amp;rememberMe=' + rememberMeVal + '&amp;top_login=' + top_loginVal;
  
                  jQuery.ajax({
                      type:'POST', 
                      url: '/myaccount/inc_login_handler.cfm', 
                      data:loginData, 
                      cache:false,
                      success: function(response){
                          var responseTrimmed = response.replace(/^\s+||\s+$/g,'');
                          if(responseTrimmed != true){    
                               jQuery("#verifying").fadeOut("slow",function(){ jQuery("#inner-dialog").fadeIn("slow"); jQuery("#login-note").html("Unable to find your login information. Please Try Again...").addClass("warning");  jQuery("#username").addClass('warning');  jQuery("#password").addClass('warning'); });
                          }else{ 
                              var successData = "&lt;center&gt;&lt;div style='padding-top:60px;'&gt;&lt;h1&gt;Success!&lt;/h1&gt;&lt;img src='/global_includes/images/checkmark.png' style='margin-top:24px; margin-bottom:20px;'&gt;&lt;br /&gt; Please Wait, Site Reloading...&lt;/div&gt;&lt;/center&gt;";
                              jQuery("#verifying").fadeOut("slow",function(){  jQuery('#verifying').html(successData).fadeIn("slow"); });
                              setTimeout(function(){ window.location.reload(); }, 1200 ); 
                          }
                      },
                      error:function (xhr, ajaxOptions, thrownError){
                          jQuery("#verifying").fadeOut("slow",function(){ jQuery("#inner-dialog").fadeIn("slow"); jQuery("#login-note").html("There was an error finding your login infomation. Please Try Again...").addClass("warning");  });
                      }  
                  });
              });
          })
      }
  }
  </code></pre>
  
  <p>I've tried this code, but can't seem to get the homepage as a logged-in user; it still returns as it would to someone who isn't logged in (my full name should appear in the header, etc.):</p>
  
  <pre><code>&gt;&gt;&gt; import requests, json
  &gt;&gt;&gt; s = requests.Session()
  &gt;&gt;&gt; page_signed_out = s.get('http://www.surfline.com/home/index.cfm')
  &gt;&gt;&gt; form_data = {'type':'POST',
  ...              'url':'/myaccount/inc_login_handler.cfm',               
  ...              'data':"username=myemail@example.com&amp;password=mypassword&amp;rememberMe=true&amp;top_login=true",
  ...              'cache':False}
  &gt;&gt;&gt; s.post(url,
  ...        data=json.dumps(form_data),
  ...        headers= {'content-type': 'application/json'})
  &lt;Response [200]&gt;
  &gt;&gt;&gt; page_signed_in = s.get('http://www.surfline.com/home/index.cfm')
  </code></pre>
  
  <p>Basically, how can I log into surfline.com from a python file, and get a page as a logged in user? I don't mind using a different library, if it is not possible with the <code>requests</code> library. Thank you.</p>
  
  <p><strong>EDIT:</strong>
  Here are the cookies after a POST has been made, as suggested by @André</p>
  
  <p>&lt;[Cookie(version=0, name='CRYPTOPASS', value='%296%25%2A%2521B%3FO%2A%3C%28', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='CRYPTOUSER', value='23%252600%2E3Z%5FM%5EEZV1IK%27TFIW%3E', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='LOGGED_OUT', value='true', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='USER_ID', value='259829', port=None, port_specified=False, domain='.surfline.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='CFID', value='437349255', port=None, port_specified=False, domain='www.surfline.com', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False), Cookie(version=0, name='CFTOKEN', value='1082d1233da237c-3E4C2D43-FFC6-4ECC-1E80D9B505E495CE', port=None, port_specified=False, domain='www.surfline.com', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=2338694583, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False)]></p>