title: Send browser headers via PHP
body: <p>How can I send a header to a website as if PHP / Apache is a browser? I'm trying to scrape a site, but it looks like they send a 404 error if it's coming from another server...</p>
  
  <p>Or, if you know any other good ways to scrape content from a site?</p>
  
  <p>Also, here is my current code:</p>
  
  <pre><code>&lt;?php
      $curl_handle=curl_init();
      curl_setopt($curl_handle,CURLOPT_URL,$_GET['url']);
      curl_setopt($curl_handle, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0)");
      curl_setopt($curl_handle, CURLOPT_REFERER, "http://google.com");
      curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);
      curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
      $buffer = curl_exec($curl_handle);
      curl_close($curl_handle);
      echo $buffer;
  ?&gt;
  </code></pre>
  
  <p>so, I'll be making an AJAX request like:</p>
  
  <pre><code>/spider.php?url=http://target.com
  </code></pre>
  
  <p>Which returns an empty string. I know this is setup right though because if i switch target with twitter.com it works... what am i missing to make this look like a full browser?</p>