title: Crawler script php
body: <p>I've grab a piece of script off here to crawl a website, put it up on my server and it works. The only issue is that if I try and crawl set the depth anything above 4 it doesn't work. I'm wondering if it due to the servers lack of resources or the code itself.</p>
  
  <pre><code>&lt;?php
  
  error_reporting(E_ALL); 
  
  function crawl_page($url, $depth)
  {
      static $seen = array();
      if (isset($seen[$url]) || $depth === 0) {
          return;
      }
      $seen[$url] = true;
  
      $dom = new DOMDocument('1.0');
      @$dom-&gt;loadHTMLFile($url);
  
      $anchors = $dom-&gt;getElementsByTagName('a');
      foreach ($anchors as $element) {
          $href = $element-&gt;getAttribute('href');
          if (0 !== strpos($href, 'http')) {
              $href = rtrim($url, '/') . '/' . ltrim($href, '/');
          }
          crawl_page($href, $depth - 1);
      }
      echo "URL:",$url,PHP_EOL;
      echo  "&lt;br/&gt;";
  }
  crawl_page("http://www.mangastream.com/", 2);
  ?&gt;
  </code></pre>
  
  <p>EDIT:</p>
  
  <p>I turned on the error reporting for the script and all I get is this </p>
  
  <p>Error 324 (net::ERR_EMPTY_RESPONSE): Unknown error.</p>