PHP / Curl: HEAD Request takes a long time on some sites <p>I have simple code that does a head request for a URL and then prints the response headers. I've noticed that on some sites, this can take a long time to complete.</p> <p>For example, requesting <code>http://www.arstechnica.com</code> takes about two minutes. I've tried the same request using another web site that does the same basic task, and it comes back immediately. So there must be something I have set incorrectly that's causing this delay.</p> <p>Here's the code I have:</p> <pre><code>$ch = curl_init(); curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt ($ch, CURLOPT_URL, $url); curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 20); curl_setopt ($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']); // Only calling the head curl_setopt($ch, CURLOPT_HEADER, true); // header will be at output curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'HEAD'); // HTTP request is 'HEAD' $content = curl_exec ($ch); curl_close ($ch); </code></pre> <p>Here's a link to the web site that does the same function: <a href="http://www.seoconsultants.com/tools/headers.asp" rel="nofollow noreferrer">http://www.seoconsultants.com/tools/headers.asp</a></p> <p>The code above, at least on my server, takes two minutes to retrieve www.arstechnica.com, but the service at the link above returns it right away.</p> <p>What am I missing?</p>