title: How to use requests.Sessions() to deliver a package to a URL with no 'action' attribute in the HTML in Python
body: <p>I want to use requests.Sessions() to deliver my login information to a website. Once logged in I want to navigate to a second URL that can only be accessed once logged in. In order to scrape data from the second URL.</p>
  <p>I am new to scraping and don't really have any experience with HTML</p>
  <p>I am working in collaboratory if that makes any difference.</p>
  <p>This is my code and the outputs:</p>
  
  <pre><code>import requests
  
  page = requests.get(&quot;https://app.gristanalytics.com/Account/Login&quot;)
  page
  </code></pre>
  <blockquote>
  <p>&lt;Response [200]&gt;</p>
  </blockquote>
  <pre><code>page.status_code
  </code></pre>
  <blockquote>
  <p>200</p>
  </blockquote>
  <pre><code>from bs4 import BeautifulSoup
  soup = BeautifulSoup(page.content, 'html.parser')
  print(soup.prettify())
  </code></pre>
  <hr />
  <p>This is the output:</p>
  <pre><code>&lt;!DOCTYPE html&gt;
  &lt;html&gt;
   &lt;head&gt;
    &lt;meta charset=&quot;utf-8&quot;/&gt;
    &lt;meta content=&quot;width=device-width, initial-scale=1, shrink-to-fit=no&quot; name=&quot;viewport&quot;/&gt;
    &lt;link href=&quot;/lib/bootstrap/css/bootstrap.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/lib/fontawesome/css/all.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/lib/datetimepicker/bootstrap-datetimepicker.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/lib/vue-multiselect/vue-multiselect.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/css/site.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;title&gt;
     Log in - Grist
    &lt;/title&gt;
   &lt;/head&gt;
   &lt;body&gt;
    &lt;div&gt;
     &lt;div class=&quot;text-center loginbox&quot;&gt;
      &lt;form method=&quot;post&quot; style=&quot;width:100%;max-width:350px;padding:15px;margin:0 auto;&quot;&gt;
       &lt;img alt=&quot;&quot; class=&quot;mb-4&quot; src=&quot;/images/grist_logo_m_black.png&quot;/&gt;
       &lt;h1 class=&quot;h3 mb-3 font-weight-normal&quot;&gt;
        Please sign in
       &lt;/h1&gt;
       &lt;div class=&quot;text-danger validation-summary-valid&quot; data-valmsg-summary=&quot;true&quot;&gt;
        &lt;ul&gt;
         &lt;li style=&quot;display:none&quot;&gt;
         &lt;/li&gt;
        &lt;/ul&gt;
       &lt;/div&gt;
       &lt;label class=&quot;sr-only&quot; for=&quot;inputEmail&quot;&gt;
        Email address
       &lt;/label&gt;
       &lt;input autofocus=&quot;&quot; class=&quot;form-control my-1&quot; data-val=&quot;true&quot; data-val-email=&quot;The Email field is not a valid e-mail address.&quot; data-val-required=&quot;The Email field is required.&quot; id=&quot;Input_Email&quot; name=&quot;Input.Email&quot; placeholder=&quot;Email address&quot; required=&quot;&quot; type=&quot;email&quot; value=&quot;&quot;/&gt;
       &lt;label class=&quot;sr-only&quot; for=&quot;inputPassword&quot;&gt;
        Password
       &lt;/label&gt;
       &lt;input class=&quot;form-control my-1&quot; data-val=&quot;true&quot; data-val-required=&quot;The Password field is required.&quot; id=&quot;Input_Password&quot; name=&quot;Input.Password&quot; placeholder=&quot;Password&quot; required=&quot;&quot; type=&quot;password&quot;/&gt;
       &lt;div class=&quot;checkbox my-3&quot;&gt;
        &lt;label&gt;
         &lt;input data-val=&quot;true&quot; data-val-required=&quot;The Remember me? field is required.&quot; id=&quot;Input_RememberMe&quot; name=&quot;Input.RememberMe&quot; type=&quot;checkbox&quot; value=&quot;true&quot;/&gt;
         Remember me
        &lt;/label&gt;
        &lt;p&gt;
         &lt;a href=&quot;/Account/ForgotPassword&quot;&gt;
          Forgot your password?
         &lt;/a&gt;
        &lt;/p&gt;
       &lt;/div&gt;
       &lt;button class=&quot;btn btn-lg btn-primary btn-block&quot; type=&quot;submit&quot;&gt;
        Sign in
       &lt;/button&gt;
       &lt;p class=&quot;mt-5 mb-3 text-muted&quot;&gt;
        © 2018-2022
       &lt;/p&gt;
       &lt;input name=&quot;__RequestVerificationToken&quot; type=&quot;hidden&quot; value=&quot;CfDJ8CxpSY-tCd5Ou0L0wqhntPACCikaoFBOUQLV0RgCaVUJgt9wRSd3p9aVswNuSLU6OPRKsbIm-qvOyZyZErcEm-E__Q2tPauexh3z_T02Oh5TZCpeY12PsUsERY3INO5LUBBmWXeUR6nG5BFHnnNdW70&quot;&gt;
        &lt;input name=&quot;Input.RememberMe&quot; type=&quot;hidden&quot; value=&quot;false&quot;/&gt;
       &lt;/input&gt;
      &lt;/form&gt;
     &lt;/div&gt;
    &lt;/div&gt;
    &lt;script src=&quot;/lib/jquery-validation/dist/Jquery.validate.min.js&quot;&gt;
    &lt;/script&gt;
    &lt;script src=&quot;/lib/jquery-validation-unobtrusive/jquery.validate.unobtrusive.min.js&quot;&gt;
    &lt;/script&gt;
   &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  <p>At this point I believe that the field names that I want to deliver the payload are:
  <strong>name=&quot;Input.Email&quot; and name=&quot;Input.Password&quot;</strong></p>
  <p>However I note that there is <strong>no action attribute in the HTML code</strong>, so I plan to send the payload to the original URL as you will see below.</p>
  <pre><code>payload = {
      'Input.Email':  'MyEmail', #yes in practice this is my actual information instead of this placeholder
      'Input.Password': 'MyPassword', #same here real password used instead
  }
  with requests.Session() as session:
    post = session.post('https://app.gristanalytics.com/Account/Login', data=payload)
    r = session.get('https://app.gristanalytics.com/Data/Brewhouse')
  soup = BeautifulSoup(r.content, 'html.parser')
  print(soup.prettify())
  </code></pre>
  <hr />
  <p>The output of this is:</p>
  <pre><code>&lt;!DOCTYPE html&gt;
  &lt;html&gt;
   &lt;head&gt;
    &lt;meta charset=&quot;utf-8&quot;/&gt;
    &lt;meta content=&quot;width=device-width, initial-scale=1, shrink-to-fit=no&quot; name=&quot;viewport&quot;/&gt;
    &lt;link href=&quot;/lib/bootstrap/css/bootstrap.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/lib/fontawesome/css/all.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/lib/datetimepicker/bootstrap-datetimepicker.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/lib/vue-multiselect/vue-multiselect.min.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;link href=&quot;/css/site.css&quot; rel=&quot;stylesheet&quot;/&gt;
    &lt;title&gt;
     Log in - Grist
    &lt;/title&gt;
   &lt;/head&gt;
   &lt;body&gt;
    &lt;div&gt;
     &lt;div class=&quot;text-center loginbox&quot;&gt;
      &lt;form method=&quot;post&quot; style=&quot;width:100%;max-width:350px;padding:15px;margin:0 auto;&quot;&gt;
       &lt;img alt=&quot;&quot; class=&quot;mb-4&quot; src=&quot;/images/grist_logo_m_black.png&quot;/&gt;
       &lt;h1 class=&quot;h3 mb-3 font-weight-normal&quot;&gt;
        Please sign in
       &lt;/h1&gt;
       &lt;div class=&quot;text-danger validation-summary-valid&quot; data-valmsg-summary=&quot;true&quot;&gt;
        &lt;ul&gt;
         &lt;li style=&quot;display:none&quot;&gt;
         &lt;/li&gt;
        &lt;/ul&gt;
       &lt;/div&gt;
       &lt;label class=&quot;sr-only&quot; for=&quot;inputEmail&quot;&gt;
        Email address
       &lt;/label&gt;
       &lt;input autofocus=&quot;&quot; class=&quot;form-control my-1&quot; data-val=&quot;true&quot; data-val-email=&quot;The Email field is not a valid e-mail address.&quot; data-val-required=&quot;The Email field is required.&quot; id=&quot;Input_Email&quot; name=&quot;Input.Email&quot; placeholder=&quot;Email address&quot; required=&quot;&quot; type=&quot;email&quot; value=&quot;&quot;/&gt;
       &lt;label class=&quot;sr-only&quot; for=&quot;inputPassword&quot;&gt;
        Password
       &lt;/label&gt;
       &lt;input class=&quot;form-control my-1&quot; data-val=&quot;true&quot; data-val-required=&quot;The Password field is required.&quot; id=&quot;Input_Password&quot; name=&quot;Input.Password&quot; placeholder=&quot;Password&quot; required=&quot;&quot; type=&quot;password&quot;/&gt;
       &lt;div class=&quot;checkbox my-3&quot;&gt;
        &lt;label&gt;
         &lt;input data-val=&quot;true&quot; data-val-required=&quot;The Remember me? field is required.&quot; id=&quot;Input_RememberMe&quot; name=&quot;Input.RememberMe&quot; type=&quot;checkbox&quot; value=&quot;true&quot;/&gt;
         Remember me
        &lt;/label&gt;
        &lt;p&gt;
         &lt;a href=&quot;/Account/ForgotPassword&quot;&gt;
          Forgot your password?
         &lt;/a&gt;
        &lt;/p&gt;
       &lt;/div&gt;
       &lt;button class=&quot;btn btn-lg btn-primary btn-block&quot; type=&quot;submit&quot;&gt;
        Sign in
       &lt;/button&gt;
       &lt;p class=&quot;mt-5 mb-3 text-muted&quot;&gt;
        © 2018-2022
       &lt;/p&gt;
       &lt;input name=&quot;__RequestVerificationToken&quot; type=&quot;hidden&quot; value=&quot;CfDJ8CxpSY-tCd5Ou0L0wqhntPAwaiYOz80Q50p5gOcDk9qSF-gR4JJpzNGOdSKiQOzcVPp8hBKgDaEwXOrbFnpgdYXkedfcnLQlXIJ1Z7HnIi5vKZybNd6VSKk_Xs5Az444e3Oug-u1UFcxq_OLX1Iu0wU&quot;&gt;
        &lt;input name=&quot;Input.RememberMe&quot; type=&quot;hidden&quot; value=&quot;false&quot;/&gt;
       &lt;/input&gt;
      &lt;/form&gt;
     &lt;/div&gt;
    &lt;/div&gt;
    &lt;script src=&quot;/lib/jquery-validation/dist/Jquery.validate.min.js&quot;&gt;
    &lt;/script&gt;
    &lt;script src=&quot;/lib/jquery-validation-unobtrusive/jquery.validate.unobtrusive.min.js&quot;&gt;
    &lt;/script&gt;
   &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  <p>Which is the same HTML as the first time, it is clear that I am not logged in, and as a result I can't get to the HTML code of the URL I want.</p>
  <p>I have tried other variations for the payload field names including:</p>
  <ol>
  <li>inputEmail (from for=)</li>
  <li>Input_Email (from id=)</li>
  <li>email (from type=)</li>
  </ol>
  <p>sample code for variation 1 would be</p>
  <pre><code>payload = {
      'inputEmail':  'MyEmail', #yes in practice this is my actual information instead of this placeholder
      'inputPassword': 'MyPassword', #same here real password used instead
  }
  </code></pre>
  <p>I get no error or warning messages when running this code so I'm a bit stuck as to what to do.</p>