Take the 2-minute tour ×

I'm trying to find the best way to configure my App instances on GAE.

I run a Saas on GWT / GAEJ. I have a small number of power-users who use the app intensively all day (rather than a large number of users who use the app for a few minutes every day).

As such I'm trying to find the best way to configure my use of Frontend Instance Classes to make it the most efficient, and give the best user experience.

I have enabled billing, and found that I need to have idle instances running to avoid latency when instances are started. I use JDO, and the when started each instances takes a long time to initialize the datastore access. So I start a number of app instances and have them running in idle mode. This gives a great user experience, but clearly means I pay for instances that just sit idle - not ideal.

I will investigate whether I can do that more efficiently.

However this is background, my real questions are as follows; If I look at the memory usage for my instances they regularly say 136MB etc (they start at around 66MB). So I imagine I have some memory leaks to find. But specifically I'd like to know: 1- I also use Memcache, presumably this memory is taken into account in the above calculation? 2- I currently use F1 instance classes, which have a memory size of 128MB. So what does it mean for my instances which seem to settle into around 136MB in size? Will they be running much more slowly as they'll be swapping out to disk the whole time? Would I be better off running a single F2 instance instead of 2 F1 instances for this reason? 3- I have found very annoyingly that GAE starts new instances even though I have 2 idle instances. This is despite me setting the min latency very high (7.5secs). I did read in the docs that this setting will have little effect when I use idle instances, but how then do I ensure that only idle instances are used, without resorting to starting new (which always results in greater latency due to the datastore initialization problems mentioned above)? (and increased cost to me by way of more instance-time)

Am I mis-understanding something? many thanks for any help.

share|improve this question

2 Answers 2

up vote 5 down vote accepted

Few notes:

a. JVM acquires the memory on-demand, but rarely releases it back to OS. This is not a memory leak.

b. You can tweak number of pending instances and response times. This gives you some control over the number of running instances. See the docs on performance settings.

c. If your code is slow to start and you don't want long response times during startup, you might want to use warmup requests.

Now on to questions:

  1. Memcache is a service external to the frontend instances and AFAIK does not store objects on the frontend instance. So the total number of objects in Memcache should not reflect on the memory used by frontend instance.

  2. The actual underlying implementation of frontend instances is not known (outside Google at least), but I doubt they use local disks for swap. You can always try to use F2 instances and see how it works out.

  3. Spin up of new instances is commanded by latency settings. If a request sits in the queue too long a new instance will be spun-up. If you instances take too long to spin up this might create a vicious circle. Try using warmup requests.

share|improve this answer
Peter, many thanks for this. I will play around with warmup requests, could well be my answer. But I am very frustrated with the instance experience I'm having. I wonder if anyone is experiencing similar issues. I tried running one F2 instead of two F1, no discernible improvement. However what is really annoying me right now is no matter how many instances I start (right now for example I have 3 idle instances) each request starts a new instance. Regardless of whether I've implemented warmup requests this should not happen right? (not when I have idle instances standing by) –  doright Mar 18 '12 at 9:53

I also had the Problem that JDO / DataNucleus initialization made the first request to an instance very slow. I use MemCache to cache almost everything that is in the Datastore. Preloading everything (for now, when data grows I will change that to preloading important things and only touch not so important objects to initialize JDO) in the warmup request solved the issue for me, that the 1 resident instance that I have only got a few request. Now the usage of my 1 resident and the other on demand instances is much better distributed - still the resident instance only gets one-third of the requests.

I think that the scheduler gets a problem with the latency calculation when the first request(s) take longer than later ones.

share|improve this answer
Hi Paul, this is very interesting, thanks very much. However I don't fully understand. Surely you don't have a read-only app? in which case the minute on of your instances needs to update the DB then it will need to incur the JDO / DataNucleus hit you mention? So isn't it just delaying the inevitable? But I am very interested in "I think that the scheduler gets a problem with the latency calculation when the first request(s) take longer than later ones." That had't occurred to me before, but sounds reasonable. –  doright May 10 '12 at 7:51
- I think JDO / DataNucleus initialization took most of the time of the first request. Now this happens in the warmup request, which apparently does not count for the latency average. - When a request starts a new instance it still takes a long time. I only see this with non-default versions, I hope the default version will always send request to existing instances until the new one is started. –  paul May 17 '12 at 17:01
okay, so regarding read / write behaviou, what you're saying is the following: put all your read requests into the cache so you don't even need to hit the database for most requests. Then in your warmup request make sure you 'touch' all of your write-to entities. That way when you actually write something it should be really quick, and yet you should get instances that actually scale up without much lag. I will try this.. –  doright May 22 '12 at 6:36

Your Answer


