title: PHP: How to get website with cURL and act like a real browser?
body: <p>There's a specific website I want to get the source code from with PHP cURL.</p>
  
  <p>Visiting this website with a bowser from my computer works without any problems.</p>
  
  <p>But when I want to access this website with my PHP script, the website recognizes that this is an automated request and shows an error message.</p>
  
  <p>This is my PHP script:</p>
  
  <pre><code>&lt;?php
  $url = "https://www.example.com";
  $user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.1 Safari/605.1.15";
  $header = array('http' =&gt; array('user_agent' =&gt; $user_agent));
  
  $ch = curl_init($url);
  curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $data = curl_exec($ch);
  curl_close($ch);
  echo $data;
  ?&gt;
  </code></pre>
  
  <p>The user agent is the same I'm also using with the browser. I'm using a local server with MAMP PRO. This means I'm using the same IP address for both, browser access and PHP script access.</p>
  
  <p>I already tried my PHP script with many different headers and options but nothing worked.</p>
  
  <p>There must be anything that makes a PHP script access look different than a browser access, for the web server I want so access the website from. But what? Do you have an idea?</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>I found out that it's working with this cURL:</p>
  
  <pre><code>curl 'https://www.example.com/' -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3' -H 'accept-language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7'
  </code></pre>
  
  <p>If I type this in e.g. the Terminal, it's showing the correct source code.</p>
  
  <p>I converted it to a PHP script as follows:</p>
  
  <pre><code>&lt;?php
  $ch = curl_init();
  
  curl_setopt($ch, CURLOPT_URL, 'https://www.example.com/');
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
  
  $headers = array();
  $headers[] = 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36';
  $headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3';
  $headers[] = 'Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7';
  curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
  
  $result = curl_exec($ch);
  curl_close($ch);
  echo $result;
  ?&gt;
  </code></pre>
  
  <p>Unfortunately, this way it's still showing the error message.</p>
  
  <p>This means, there must be anything that makes a command line access look different than a browser access, for the web server I want so access the website from. But what is it?</p>