title: Python to get/retrieve data from web Application
body: <p>I am trying to automate a reporting process with Python 
  The Idea is to login to the page and then get the URL :
  <a href="https://XXXXXXXXXXXXXXXXXXXXXX/#dashboard/1" rel="nofollow">https://XXXXXXXXXXXXXXXXXXXXXX/#dashboard/1</a>  as you can see in the screen shot :
  but if you click view source it is here is the Html code (Na thing on it ):</p>
  
  <pre><code>&lt;!DOCTYPE html&gt;
  &lt;html&gt;
  &lt;head&gt; 
  &lt;meta charset="UTF-8" /&gt; 
  &lt;title&gt;XXXXXXXXXXXXXXXX&lt;/title&gt; 
  &lt;link id="css-link" rel="stylesheet" href="css/app_108976.css" /&gt; 
     &lt;link rel="apple-touch-icon" sizes="57x57" href="img/favicons/apple-touch-   icon-57x57.png" /&gt; 
    &lt;link rel="apple-touch-icon" sizes="60x60" href="img/favicons/apple-touch-icon-60x60.png" /&gt; 
   &lt;link rel="icon" type="image/png" href="img/favicons/favicon-32x32.png" sizes="32x32" /&gt; 
   &lt;link rel="icon" type="image/png" href="img/favicons/favicon-16x16.png" sizes="16x16" /&gt; 
   &lt;link rel="manifest" href="img/favicons/manifest.json" /&gt; 
   &lt;meta name="msapplication-TileColor" content="#2b5797" /&gt; 
   &lt;meta name="theme-color" content="#ffffff" /&gt; 
   &lt;!--[if lte IE 9]&gt;
       &lt;link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css"&gt;
      &lt;![endif]--&gt; 
      &lt;!--[if lte IE 8]&gt;
        &lt;style type="text/css"&gt;
             #browser-not-supported-container {
              display: block;
              text-align: left;
          }
  
           #login-panel-content {
              display: none;
          }
  
          #browser-not-supported-link {
              display: block;
              padding-top: 10px;
          }
  
            #browser-not-supported-download-link {
              text-decoration: underline;
          }
      &lt;/style&gt;
      &lt;![endif]--&gt; 
   &lt;/head&gt; 
   &lt;body&gt; 
  &lt;div id="feature-detection"&gt;&lt;/div&gt; 
  &lt;div id="main-page"&gt; 
  &lt;noscript&gt; 
  &lt;div class="page login-page"&gt; 
   &lt;div id="page-header" class="page-header"&gt; 
    &lt;div class="page-header-logo-container"&gt; 
     &lt;img class="page-header-logo" src="img/XXXXXXXXXXXXXXXXXXX.png" /&gt; 
    &lt;/div&gt;
    &lt;!--
                   --&gt;
    &lt;div class="user-menu"&gt;&lt;/div&gt; 
    &lt;div id="page-header-navbar" class="page-header-navbar"&gt;&lt;/div&gt; 
    &lt;/div&gt; 
    &lt;div class="page-content"&gt; 
    &lt;div id="login-container" class="login-container hidden"&gt; 
     &lt;div class="login-container-top"&gt; 
      &lt;div id="login-panel" class="login-panel"&gt; 
       &lt;div id="login-panel-header" class="login-panel-header"&gt;&lt;/div&gt; 
       &lt;style type="text/css"&gt;
                                      #login-panel-content {
                                          display: none;
                                      }
                                  &lt;/style&gt; 
       &lt;div class="no-script-container"&gt; 
        &lt;div class="no-script-content"&gt; 
         &lt;div class="no-script-header"&gt;
          JavaScript Disabled
           &lt;/div&gt; JavaScript is required to use this application.  
        &lt;/div&gt; 
       &lt;/div&gt; 
      &lt;/div&gt; 
     &lt;/div&gt; 
    &lt;/div&gt; 
    &lt;/div&gt; 
    &lt;div class="page-footer"&gt;&lt;/div&gt; 
    &lt;/div&gt; 
    &lt;/noscript&gt; 
    &lt;script&gt;
              (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
                          (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
                      m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
              })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
  
              ga('create', 'UA-6177009-1', 'auto', {'allowLinker': true});
              ga('require', 'linker');
              ga('linker:autoLink', ['www.XXXXXXXXXXXr.com'] );
              ga('send', 'pageview');
  
          &lt;/script&gt; 
         &lt;script type="text/javascript"&gt;
              /* &lt;![CDATA[ */
              var google_conversion_id = 973777747;
              var google_custom_params = window.google_tag_params;
              var google_remarketing_only = true;
              /* ]]&gt; */
           &lt;/script&gt; 
          &lt;script type="text/javascript"     src="//www.googleadservices.com/pagead/conversion.js"&gt;
          &lt;/script&gt; 
          &lt;noscript&gt; 
         &lt;div style="display:inline;"&gt; 
         &lt;img height="1" width="1" style="border-style:none;" alt="" src="//googleads.g.doubleclick.net/pagead/viewthroughconversion/973777747/?value=0&amp;amp;guid=ON&amp;amp;script=0" /&gt; 
       &lt;/div&gt; 
       &lt;/noscript&gt; 
       &lt;/div&gt; 
       &lt;script id="bootstrapper" src="js/main_108976.js"&gt;&lt;/script&gt;   
       &lt;/body&gt;
       &lt;/html&gt;
  </code></pre>
  
  <p>Here is the Python code that I am using :</p>
  
  <pre><code>       import requests
         url2 ="https://admin.XXXXXXXXXXXXX.net/#exec-reports"
         payload +{"apiKey":"8770XXXXXX8t8","username":"XXXXXXXXXXXXX.net","password":"XXXXXXXXX","timestamp":"1449666522626"}
       with requests.Session() as Req:
             url ="https://admin.XXXXXXXXX.net/XXXXXXapi/v1/authenticatedSession"
             Req.post(url, data=payload)
             Response = Req.get(url2)
             print (Response.headers)
             print (Response.status_code)
             print(Response.text)
  </code></pre>
  
  <p>and here is the output :
      {'Accept-Ranges': 'bytes', 'ETag': 'W/"4290-1449645452000"', 'Last-Modified': 'Wed, 09 Dec 2015 07:17:32 GMT', 'Vary': 'Accept-Encoding', 'Server': 'XXXXXX', 'Transfer-Encoding': 'chunked', 'Date': 'Mon, 14 Dec 2015 09:09:20 GMT', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html;charset=UTF-8', 'X-FRAME-OPTIONS': 'SAMEORIGIN'}
  200
  and the same HTML that is posted above.
  the question is  do you have any idea how can i retrive the data that i need  ? i can post more information if needed 
  thanks a lot </p>