title: How to handle javascript content and redirects after successful weblogin SSO authentication?
body: <p>I am writing a python script that downloads class content(mp4, pdf) from my school website. My school uses Weblogin SSO authentication to access any of their protected urls.</p>
  
  <p>I was able to authenticate my credentials using the first part of the script below:</p>
  
  <pre><code>#1. Authenticate
  login_url = "https://weblogin.MY_SCHOOL.edu/login"
  payload = {'login':'my_loging','password':'my_pass'}
  target_url = "https://My_SCHOOL.instructure.com/courses/12345678""
  
  with requests.Session() as c: 
      req_headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36}'}
      c.headers.update(req_headers)
      c.get(login_url) # to get cookies
      c.post(url1, data=payload) #,headers = req_headers)
  
  #2. get html from target site
      W1 = c.get(target_url)
      print(W1.url)
      print(W1.text)
  
  #3. parse html and download content.
      #tbc
  </code></pre>
  
  <p>I can see that my authentication was successful in c.post.text, but whenver I try to access any of the target sites using get() in same requests.session(), I don't get the expected html content for my class, but rather a message that reads:</p>
  
  <blockquote>
    <p>"Since your browser does not support JavaScript, you must press the
    Continue button once to proceed"</p>
  </blockquote>
  
  <p>And the target URL redirects to this url:</p>
  
  <blockquote>
    <p>"<a href="https://idp.MY_SCHOOL.edu/idp/profile/SAML2/Redirect/SSO" rel="nofollow">https://idp.MY_SCHOOL.edu/idp/profile/SAML2/Redirect/SSO</a>"</p>
  </blockquote>
  
  <p>Why am I not able to access the target url(s) after a successful SSO authentication? I am not sure if javascript support in the requests module is the issue here because even when I disable JS support in my internet browser, I am able to see some html content of the target_url, albeit not all of it. It seems strange that my get() request gets stuck in the redirected url: "https:.../SAML2/Redirect/SSO"</p>
  
  <p>I would appreciate any pointers on how to go around this issue. I wouldn't want to use webdrivers such as selenium or mechanize. I have used QtWebkit to render Javascript content, but I don't know if it is even possible to transfer my  authentication cookies from my request.session() to QtWebkit.</p>
  
  <p>Any help is much appreciated. Thanks  </p>