title: Does Instagram return HTTP 500 response due to rate limiting and/or some form of request filtering?
body: <p>I'm developing a tool for checking the integrity of links in a web page.</p>
  
  <p>I've noticed that various Instagram URLs will return a HTTP 500 response in cases where if one were to visit the given URL in a browser one would get a HTTP 200 response accompanied by the expected resource.</p>
  
  <p>This is when requesting regular Instagram URLs as one would do as a browser user and not when using the REST API.</p>
  
  <p>A typical request/response using cURL:</p>
  
  <pre><code>curl -v http://instagram.com/p/YKTkxHBA-P/
  * About to connect() to instagram.com port 80 (#0)
  *   Trying 54.225.159.246... connected
  &gt; GET /p/YKTkxHBA-P/ HTTP/1.1
  &gt; User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
  &gt; Host: instagram.com
  &gt; Accept: */*
  &gt; 
  &lt; HTTP/1.1 500 Server Error
  &lt; Cache-Control: no-cache
  &lt; Content-Type: text/html
  &lt; Date: Tue, 15 Oct 2013 08:31:09 GMT
  &lt; Server: nginx
  &lt; Content-Length: 87
  &lt; Connection: keep-alive
  &lt; 
  &lt;html&gt;&lt;body&gt;&lt;h1&gt;500 Server Error&lt;/h1&gt;
  An internal server error occured.
  &lt;/body&gt;&lt;/html&gt;
  * Connection #0 to host instagram.com left intact
  * Closing connection #0
  </code></pre>
  
  <p>I did for some time get HTTP 200 responses in such cases but am now consistently getting HTTP 500 responses.</p>
  
  <p>This is all specific to a given host; such URLs, even when sending requests with cURL, will return HTTP 200 responses from other machines.</p>
  
  <p>Due to this being specific to the host sending the requests I suspect a form of rate limiting or request filtering by IP is going on, however I can find no documentation to the effect.</p>
  
  <p>Will Instagram officially return a HTTP 500 response as above due to a given IP being denied access?</p>