title: 404 vs 403 when directory index is missing
body: <p>This is mostly a philosophical question about the best way to interpret the HTTP spec. Should a directory with no directory index (e.g. index.html) return 404 or 403? (403 is the default in Apache.)</p>
  
  <p>For example, suppose the following URLs exist and are accessible:</p>
  
  <pre><code>http://example.com/files/file_1/
  http://example.com/files/file_2/
  </code></pre>
  
  <p>But there's nothing at:</p>
  
  <pre><code>http://example.com/files/
  </code></pre>
  
  <p>(Assume we're using 301s to force trailing slashes for all URLs.)</p>
  
  <p>I think several things should be taken into account:</p>
  
  <ul>
  <li>By default, Apache returns 403 in this scenario. That's significant to me. They've thought about this stuff, and they made the decision to use 403.</li>
  <li>According to W3C, 403 means "The server understood the request, but is refusing to fulfill it." I take that to mean you should return 403 if the URL is meaningful but nonetheless forbidden.</li>
  <li>403 might result in information disclosure if the client correctly guesses that the URL maps to a real directory on disk.</li>
  <li><code>http://example.com/files/</code> isn't a resource, and the fact that it internally maps to a directory shouldn't be relevant to the status code.</li>
  <li>If you interpret the URL scheme as defining a directory structure <em>from the client's perspective</em>, the internal implementation is still irrelevant, but perhaps the outward appearance should indeed have some bearing on the status codes. Maybe, even if you created the same URL structure without using directories internally, you should still use 403s, because it's about the client's perception of a directory structure.</li>
  </ul>
  
  <p>In the balance, what do you think is the best approach? Should we just say "a resource is a resource, and if it doesn't exist, it's a 404?" Or should we say, "if it has slashes, it looks like a directory to the client, and therefore it's a 403 if there's no index?"</p>
  
  <p>If you're in the 403 camp, do you think you should go out of your way to return 403s even if the internal implementation doesn't use directories? Suppose, for example, that you have a dynamic web app with this URL: <code>http://example.com/users/joe</code>, which maps to some code that generates the profile page for Joe. Assuming you don't write something that lists all users, should <code>http://example.com/users/</code> return 403? (Many if not all web frameworks return 404 in this case.)</p>