title: Asp.net randomly stops obeying forms authentication whitelist
body: <h2>The problem</h2>
  
  <p>Last month we move our asp.net website farm from Server 2008 R2 to Server 2012 R2 and upgraded to asp.net 4.5. We are using cookied forms authentication to prevent unauthorized access to the website.</p>
  
  <pre><code>&lt;authorization&gt;
    &lt;deny users="?" /&gt;
    &lt;allow users="*" /&gt;
  &lt;/authorization&gt;
  </code></pre>
  
  <p>We have certain assets and pages (ex: sign in page) that are whitelisted in the web.config: </p>
  
  <pre><code>&lt;location path="signin"&gt;
     &lt;system.web&gt;
       &lt;authorization&gt;
         &lt;allow users="*" /&gt;
       &lt;/authorization&gt;
     &lt;/system.web&gt;
  &lt;/location&gt;
  </code></pre>
  
  <p>Over the last few months we've been noticing that IIS/Asp.net randomly stops obeying the whitelist and assume everything needs to be authenticated. All requests to the site on that server will be redirected to the signin page which then throws a 500 error. No whitelisted assets can be retrieved. </p>
  
  <p>There are then 2 errors in the event viewer that we can see when IIS is messed up. The first:</p>
  
  <pre><code>Exception type: NullReferenceException 
      Exception message: Object reference not set to an instance of an object.
     at System.Web.PipelineModuleStepContainer.GetNextEvent(RequestNotification notification, Boolean isPostEvent, Int32 eventIndex)
     at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error)
     at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb)
     at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context)
  </code></pre>
  
  <p>This second one doesn't show up all the time: </p>
  
  <pre><code>Event code: 4005 
  Event message: Forms authentication failed for the request. Reason: The ticket supplied has expired. 
  </code></pre>
  
  <p>The iis process will be working find for hours then all the sudden start doing this weirdness. As soon as we recycle the app pool, or even just modify the web.config the site starts working again. </p>
  
  <h2>Things we've tried</h2>
  
  <p>Honestly we are quite stumped. This wasn't happening on our old servers, but we've made quite a few changes to our site since then but nothing related to authentication. </p>
  
  <ol>
  <li><p>We are in a webfarm and we define our machine key inside of our web.config. </p>
  
  <pre><code>&lt;machineKey validationKey="XXX" decryptionKey="XXX" validation="SHA1" decryption="AES" /&gt;
  </code></pre></li>
  <li><p>We are targeting asp.net 4.5</p>
  
  <pre><code>&lt;httpRuntime targetFramework="4.5" executionTimeout="120" maxQueryStringLength="4096"  minFreeThreads="72" minLocalRequestFreeThreads="88"  maxRequestLength="32768" /&gt;
  </code></pre></li>
  <li><p>We recreated the application pool within IIS.</p></li>
  <li>Not sure if it matters but we use IIS Shared Config and shared certificates.  </li>
  <li>The issue is happening on all of the web servers in the farm, not just one. </li>
  <li>We reinstalled the OS on one of the servers yesterday... so we'll see if that fixes anything. </li>
  <li>It doesn't seem to be tied to memory usage. Sometimes iis is only using 4gb, sometimes 6gb. </li>
  <li>It doesn't seem to be tied to a certain page execution that we can tell. </li>
  <li>I've run debug diag against a memory dump and there aren't any threads that are running long nor crazy memory usage. </li>
  </ol>
  
  <p>Yea, we are stumped. Any help is appreciated. </p>