title: Web Crawler issue in perl
body: <p>I had built a web crawler in Perl. </p>
  
  <p>I am using </p>
  
  <pre><code>HTML::ContentExtractor
  
  LWP::UserAgent 
  
  HTML::LinkExtor
  </code></pre>
  
  <p>to extract text form webpages.</p>
  
  <p>Reference link for sample code <a href="https://stackoverflow.com/questions/19157269/perl-print-matched-content-only">web cralwer perl</a></p>
  
  <p><strong>Issue:</strong></p>
  
  <p>The issue is that it does not get text from web pages that have the extension as <code>.aspx</code> .
  It works perfectly for other webpages.I could not figure out the issue why this crawler fails for <code>aspx</code> pages.</p>