<p>It is the response of the website that don't allow empty user agents:</p>

<pre><code>HTTP/1.1 403 Forbidden

&lt;!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"&gt;
&lt;html&gt;
&lt;head&gt;&lt;title&gt;403 Forbidden&lt;/title&gt;&lt;/head&gt;
&lt;body&gt;
&lt;h1&gt;Access Denied&lt;/h1&gt;

 &lt;p&gt;Sadly, your client does not supply a proper User-Agent,
 and is consequently excluded.&lt;/p&gt;
 &lt;p&gt;We have an inordinate number of problems with automated scripts
 which do not supply a User-Agent, and violate the automated access
 guidelines posted at arxiv.org
 -- hence we now exclude them all.&lt;/p&gt;
 &lt;p&gt;(In rare cases, we have found that accesses through proxy servers
 strip the User-Agent information. If this is the case, you need to contact
 the administrator of your proxy server to get it fixed.)&lt;/p&gt;


&lt;p&gt;If you believe this determination to be in error, see
&lt;b&gt;http://arxiv.org/denied.html&lt;/b&gt; for additional information.&lt;/p&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre>

<p>If you use for example the user agent "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko" in your request, it will work:</p>

<pre><code>$options = array(
  'http'=&gt;array(
    'method'=&gt;"GET",
    'header'=&gt;"User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko\r\n"
  )
);
$context = stream_context_create($options);
$str = file_get_contents($url, false, $context);
</code></pre>
Extracting the title and abstract from a webpage
<p>I am trying to extract the title and abstract from arXiv pages, for example <a href="http://arxiv.org/abs/1207.0102" rel="nofollow">http://arxiv.org/abs/1207.0102</a>, my code currently looks like</p>

<pre><code>function get_title($url){
  $str = file_get_contents($url);
  if(strlen($str)&gt;0){
    $str = trim(preg_replace('/\s+/', ' ', $str)); // supports line breaks inside &lt;title&gt;
    preg_match("/\&lt;title\&gt;(.*)\&lt;\/title\&gt;/i",$str,$title); // ignore case
    return $title[1];
  }
}

echo get_title("http://arxiv.org/abs/1207.0102");
</code></pre>

<p>When I run this code, this error comes up</p>

<blockquote>
  <p>Warning: file_get_contents(<a href="http://arxiv.org/abs/1207.0102" rel="nofollow">http://arxiv.org/abs/1207.0102</a>): failed to
  open stream: HTTP request failed! HTTP/1.1 403 Forbidden in
  C:\wamp\www\mysite\Index.php</p>
</blockquote>

<p>This problem doesn't happen when I try different urls for example <a href="http://www.washingtontimes.com/" rel="nofollow">http://www.washingtontimes.com/</a>.</p>

<p>Does anyone know why this happens?</p>

<p>Also, is it possible to extract the abstract from this webpage?</p>