title: Python beautifulsoup print does not print whole html page
body: <p>I am scraping a website that has a pagination in it. I was testing the loop and print the output in it from beautifulsoup. When the results are printed, I noticed that the result is not a complete html text. It only includes the first part of the html. Here are my code</p>
  
  <pre><code>from bs4 import BeautifulSoup
  import requests
  import time
  total_pages = 2295
  
  
  for i in range(1,total_pages,1):
      pageNumber = str(i)
      url = requests.get("https://www.propertyguru.com.sg/property-for-sale/"+pageNumber+"?order=desc&amp;property_type=N&amp;property_type_code%5B0%5D=CONDO&amp;property_type_code%5B1%5D=APT&amp;property_type_code%5B2%5D=WALK&amp;property_type_code%5B3%5D=CLUS&amp;property_type_code%5B4%5D=EXCON&amp;sort=date").text
      soup = BeautifulSoup(url,'html.parser')
  
    print(soup.prettify())
  </code></pre>
  
  <p>When i print <code>soup.prettify()</code> the result is this</p>
  
  <pre><code>&lt;!DOCTYPE doctype html&gt;
  &lt;!--[if gt IE 9]&gt;&lt;!--&gt;
  &lt;html class="no-js is-new-brand" lang="en"&gt;
   &lt;!--&lt;![endif]--&gt;
   &lt;head&gt;
    &lt;title&gt;
    &lt;/title&gt;
    &lt;meta charset="utf-8"/&gt;
    &lt;meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/&gt;
    &lt;meta content="width=device-width, initial-scale=1" name="viewport"/&gt;
    &lt;meta content="app-id=482524585" name="apple-itunes-app"&gt;
     &lt;meta content="app-id=com.allproperty.android.consumer.sg" name="google-play-app"&gt;
      &lt;meta content="9iVXbwdOPHOH_byBFBScAHm5x-kvcPzBS_fJBFPBwbo" name="google-site-verification"&gt;
       &lt;meta content="46acd457be6effa0" name="y_key"/&gt;
       &lt;meta content="893837EF69C47405FBAFAB120889A598" name="msvalidate.01"/&gt;
       &lt;link href="/images/is-new-brand-favicon.ico" rel="SHORTCUT ICON"/&gt;
       &lt;link href="/search.xml" rel="search" title="PropertyGuru Search" type="application/opensearchdescription+xml"/&gt;
       &lt;link href="https://cdn.pgimgs.com/1574318624/sf2-search/bundles/guruweblayout/img/is-new-brand-touch-logo.png" rel="apple-touch-icon"/&gt;
       &lt;link href="https://cdn.pgimgs.com/1574318624/sf2-search/bundles/guruweblayout/img/is-new-brand-touch-logo.png" rel="android-touch-icon"/&gt;
       &lt;script&gt;
        // check for browsers without complete flex support ( &lt; IE 10)
                  window.onload = function(e){
                          if(Function('/*@cc_on return document.documentMode&lt;=10@*/')()) {
                                  window.location = '/ie-notsupported';
                          }
                  };
       &lt;/script&gt;
       &lt;link href="//cdn1.pgimgs.com/1574318624/sg-static/cssprod/propertyguru/layout.css" rel="stylesheet" type="text/css"/&gt;
       &lt;link href="//cdn1.pgimgs.com/1574318624/sg-static/cssprod/propertyguru/sg.css" rel="stylesheet" type="text/css"/&gt;
       &lt;link href="//cdn1.pgimgs.com/1574318624/sg-static/cssprod/propertyguru/new_styles.css" rel="stylesheet" type="text/css"/&gt;
       &lt;script src="//cdn1.pgimgs.com/1574318624/sg-static/jsprod/lib/modernizr-custom.min.js" type="text/javascript"&gt;
       &lt;/script&gt;
       &lt;script src="//cdn1.pgimgs.com/1574318624/sg-static/jsprod/jquery-1.12.3.min.js" type="text/javascript"&gt;
       &lt;/script&gt;
       &lt;script type="text/javascript"&gt;
        var guruApp = {"environment":null,"widgetSearch":null,"widgetPoll":null,"widgetGoogleAnalytics":{"dimensions":{"dimension3":"Production","dimension4":"en","dimension13":"SG","dimension14":"web"},"googleAnalyticsObject":null,"config":{"trackingId":"UA-2417512-2","cookieDomain":"propertyguru.com.sg","siteSpeedSampleRate":10}},"userSession":{"user":{"id":null,"username":null,"roles":null,"shortlist":0,"beta":false}},"isResponsive":"false","identityEndpoint":"https:\/\/identity.propertyguru.com\/identity","defaultCurrency":"SGD","googleMaps":{"key":"AIzaSyBlCo7kpcBszvIZoH709avg1rmUjjiop0k"},"googleApis":{"key":"367223124563-is5hdjeal1rr7og4i8ii7t8imihr1dg1.apps.googleusercontent.com"}};
       &lt;/script&gt;
       &lt;link href="https://fonts.googleapis.com/css?family=Roboto:400,500" rel="stylesheet" type="text/css"/&gt;
       &lt;link href="https://fonts.googleapis.com/css?family=Nunito:600" rel="stylesheet" type="text/css"/&gt;
       &lt;!--[if gt IE 8]&gt;&lt;!--&gt;
       &lt;link href="https://cdn.pgimgs.com/1574318624/sf2-search/css/legacy_css.css" rel="stylesheet" type="text/css"&gt;
        &lt;link href="//cdn1.pgimgs.com/1574318624/sg-static/cssprod/rich/fixes.css" rel="stylesheet" type="text/css"&gt;
         &lt;!--&lt;![endif]--&gt;
         &lt;script type="text/javascript"&gt;
          &lt;!--
          var GMAP_KEY = "AIzaSyCUbmYAT3lyhBvao9Yg-WsKtRbMxO-VvVQ";
          var REGION = "SG";
      var images = [];
      var freetextUrl = '//api.propertyguru.com/v1/autocomplete?limit=10&amp;locale=en&amp;format=csv_legacy&amp;region=sg&amp;objectType=HDB_ESTATE,DISTRICT,PROPERTY,STREET,MRT_STATION,SCHOOL';
          //--&gt;
         &lt;/script&gt;
         &lt;!-- GOOGLE AD MANAGER --&gt;
         &lt;div class="clearboth"&gt;
         &lt;/div&gt;
         &lt;!-- Begin comScore Tag --&gt;
         &lt;script&gt;
          var _comscore = _comscore || [];
    _comscore.push({ c1: "2", c2: "13151479" });
    (function() {
      var s = document.createElement("script"), el = document.getElementsByTagName("script")[0]; s.async = true;
      s.src = (document.location.protocol == "https:" ? "https://sb" : "http://b") + ".scorecardresearch.com/beacon.js";
      el.parentNode.insertBefore(s, el);
    })();
         &lt;/script&gt;
         &lt;noscript&gt;
          &lt;img src="https://sb.scorecardresearch.com/p?c1=2&amp;amp;c2=13151479&amp;amp;cv=2.0&amp;amp;cj=1"/&gt;
         &lt;/noscript&gt;
         &lt;!-- End comScore Tag --&gt;
         &lt;!-- GOOGLE ANALYTICS CODE --&gt;
         &lt;script src="https://cdn.pgimgs.com/1574318624/sf2-search/bundles/guruweblayout/js/desktop/logger.js" type="text/javascript"&gt;
         &lt;/script&gt;
         &lt;script src="https://cdn.pgimgs.com/1574318624/sf2-search/bundles/guruweblayout/js/fingerprint2.min.js" type="text/javascript"&gt;
         &lt;/script&gt;
         &lt;script src="https://cdn.pgimgs.com/1574318624/sf2-search/bundles/guruwidget/js/desktop/jquery.widgetGoogleAnalytics.js" type="text/javascript"&gt;
         &lt;/script&gt;
         &lt;!-- Google Analytics --&gt;
         &lt;script type="text/javascript"&gt;
          (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
          (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
              m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
          })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
         &lt;/script&gt;
         &lt;script type="text/javascript"&gt;
          if (typeof guruApp != 'undefined' &amp;&amp; typeof guruApp.widgetGoogleAnalytics != 'undefined' &amp;&amp; guruApp.widgetGoogleAnalytics.googleAnalyticsObject != null) {
      guruApp.widgetGoogleAnalytics.googleAnalyticsObject.init();
  }
         &lt;/script&gt;
         &lt;script src="https://cdn.pgimgs.com/1574318624/sf2-search/bundles/guruweblayout/js/desktop/jquery.eventDispatcher.js" type="text/javascript"&gt;
         &lt;/script&gt;
         &lt;script type="text/javascript"&gt;
          $(document).ready(function () {
                  var $body = $('body'),
                      track = function(category, action, label, value, noninteraction, dimensions) {
                          label = cleanText(label);
                          guruApp.widgetGoogleAnalytics.googleAnalyticsObject.trackEvent(category, action, label, value, noninteraction, dimensions);
                      },
                      cleanText = function(str) {
                          return str.replace(/^https?:\/\/[^\/]+/, '').replace(/^\s+/, '').replace(/\s+$/, '').replace(/\s+/, ' ');
                      };
                  $body.find('.dropdown .dropdown-menu li.mainnav-areainsider').click(function () {
                      $body.trigger('ga.mainnav.areainsider.click');
                  });
              });
         &lt;/script&gt;
         &lt;!-- ELOQUA TRACKING CODE --&gt;
         &lt;script type="text/javascript"&gt;
          var _elqQ = _elqQ || [];
      _elqQ.push(['elqSetSiteId', '659351510']);
      _elqQ.push(['elqTrackPageView']);
  
      (function () {
          function async_load() {
              var s = document.createElement('script'); s.type = 'text/javascript'; s.async = true;
              s.src = '//img03.en25.com/i/elqCfg.min.js';
              var x = document.getElementsByTagName('script')[0]; x.parentNode.insertBefore(s, x);
          }
          if (window.addEventListener) window.addEventListener('DOMContentLoaded', async_load, false);
          else if (window.attachEvent) window.attachEvent('onload', async_load);
      })();
         &lt;/script&gt;
         &lt;script defer="" src="/pg186791.js" type="text/javascript"&gt;
         &lt;/script&gt;
         &lt;style type="text/css"&gt;
          #d__fFH{position:absolute;top:-5000px;left:-5000px}#d__fF{font-family:serif;font-size:200px;visibility:hidden}#weeawqsxdstyxxvz{display:none!important}
         &lt;/style&gt;
        &lt;/link&gt;
       &lt;/link&gt;
      &lt;/meta&gt;
     &lt;/meta&gt;
    &lt;/meta&gt;
   &lt;/head&gt;
   &lt;body class="web_filter_recaptcha SG-web_filter_recaptcha layout-web lang-en app-sg legacy is-new-brand" id="web_filter_recaptcha"&gt;
    &lt;div id="wrapper-outer"&gt;
     &lt;div id="wrapper"&gt;
      &lt;div id="wrapper-inner"&gt;
       &lt;div class="alert alert-warning" id="gdpr-alert" role="alert" style="margin-bottom: 0; display:none;"&gt;
        To comply with GDPR we will not store any personally identifiable information from you. Therefore we will serve sub-optimal experience where some features such as Login/Signup are disabled. However, you will be able to search and see all the properties, see agent contact details and contact them offline on your own.
       &lt;/div&gt;
       &lt;header class="navbar navbar-default" id="navbar-main"&gt;
        &lt;div class="header-bg"&gt;
         &lt;div class="container"&gt;
          &lt;nav class="header-nav clearfix" role="navigation"&gt;
           &lt;div class="navbar-header"&gt;
            &lt;button class="navbar-toggle" type="button"&gt;
             &lt;span class="sr-only"&gt;
              Toggle navigation
             &lt;/span&gt;
             &lt;i class="pgicon pg
  
  &lt;!DOCTYPE doctype html&gt;
  &lt;!--[if gt IE 9]&gt;&lt;!--&gt;
  &lt;html class="no-js is-new-brand" lang="en"&gt;
   &lt;!--&lt;![endif]--&gt;
   &lt;head&gt;.....AND SO ON AND SO FOURTH
  
  </code></pre>
  
  <p>It only prints some contents but not the whole html contents. </p>