title: 503 Over Quota Every morning
body: <p>Every morning between 8 and 9 am CET my site shows this error:</p>
  
  <pre><code>Error Over Quota
  This application is temporarily over its serving quota. Please try again later.
  </code></pre>
  
  <p>although billing is enabled and the site has been running for many months now. On the old billing status page, there's a setting for <code>Maximum Daily Budget</code> (Set this to handle peak traffic and to buffer against sudden traffic surges.) which is set to $0.00, but even if I change that to e.g. $10.00, it still shows the 503 error, so it seems this has nothing to do with it.</p>
  
  <p>The new billing page looks like this:</p>
  
  <p><img src="https://i.stack.imgur.com/6eYbq.png" alt="New Billing Page"></p>
  
  <p>It happens every morning between 7 and 8 am CET, so that would be around midnight PST which would at least indicate that it still might have something to do with billing?</p>
  
  <p>Here's how the external monitoring system shows the outages, i.e. multiple outages every morning and then no problems for the rest of the day.</p>
  
  <p><img src="https://i.stack.imgur.com/naUHS.png" alt="Pingdom"></p>
  
  <p>The Google Developers Console Overview page <code>Errors by status code</code> also shows the 503 error in green.</p>
  
  <p><img src="https://i.stack.imgur.com/I9AAv.png" alt="Errors by status code"></p>
  
  <p>If I look at the monitoring logs most of the 503 errors between 7 and 9am for the following pages:</p>
  
  <ul>
  <li>/</li>
  <li>/wp-cron.php</li>
  </ul>
  
  <p>And then there are some 500 errors, e.g. at 8 am </p>
  
  <pre><code>08:07:08.092 ... [26/Nov/2014:23:07:08 -0800] "GET / HTTP/1.1" 500 0 - "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)" "www.coworking-radolfzell.de" ms=921 cpu_ms=1042 loading_request=1 exit_code=108 instance=00c61b117c9ad93b9da1f63314065ff8f4188095 app_engine_release=1.9.16
  
  08:07:08.092 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application.
  
  08:07:08.092 Process terminated due to exceeding quotas.
  </code></pre>
  
  <p>The (new) <code>quota details</code> page says <code>Quotas are reset every 24 hours. Next reset: 1 hours</code> and now it's 8:30 am CET. All of the listed resources are rated <code>Okay</code>.</p>
  
  <p>If I go to the old <code>Quota Details</code>p page, though, I see that my requests' <code>Frontend Instance Hours</code>have reached 100% and are rated <code>Limited</code>.</p>
  
  <p><img src="https://i.stack.imgur.com/qsdDt.png" alt="Frontend Instance Hours"></p>
  
  <p>So I went back to the new console under <code>Compute / App Engine / Settings</code> which showed <code>Your budget today is $10.00. Effective tomorrow, you will be using only free quota.</code> which I now increased to a daily budget of USD 5.</p>
  
  <p><img src="https://i.stack.imgur.com/uxSoy.png" alt="Application Settings"></p>
  
  <p>My site now still shows the 503 error, but hopefully this will be the solution. I should be able to tell in 24 hours, shouldn't I?</p>
  
  <p>If anyone from Google App Engine is reading this, there might be some inconsistencies between the old and new panels that should probably be fixed.</p>