I'm running a Ruby on Rails application with Passenger + Apache. There's also a perl mailing list server (Sympa) running with mod_perl.

I have root and sh access to the server (an Ubuntu machine).

Most of the times it runs just fine; the server has more than enough memory and disk space to handle the workload.

But every now and then (maybe once every 3 months or so) some Apache process seems to "go rogue" and starts consuming memory and cpu like crazy. As a result, the website goes down (sometimes I had to do a cold reboot; just rebooting Apache wasn't enough). I've done some tests and it seems that the processes being hanged out are Ruby processes, which makes me suspect that Passenger is missbehaving.

Here are some Munin graphs for the last "outage" (which just happened)

I'm suspicious about the ruby process (Passenguer). However, I can't pinpoint the conditions that trigger the "wild process" (I only host the app, I didn't develop it). Although it would be nice, it might be simpler to just "kill the hanging process".

I'd like to know if there's a way to "auto-kill" processes that take too much memory or CPU. Is there any Apache module or configuration flag that deals with this problem?

EDIT: So far I've tried Apache's RLimitCPU, RLimitMEM and RLimitNPROC to no avail - its seems that Passenger is impervious to these settings. I'm pretty sure the issue is on Passenger now, so I'm re-tagging and re-writing parts of this question.

  • Are there any cron jobs that are running every hour? – Raffael Luthiger Jun 15 '11 at 12:01
  • There are some hourly cron jobs. However, the outage seems to happen every 3 months or so. It's more likely a crawler issue (too many requests that take long to execute in too little time) – egarcia Jun 15 '11 at 12:03
  • If it is really too many requests then you can see it by analysing the number of open connections (netstat). – Raffael Luthiger Jun 15 '11 at 13:37

ulimits (configured through /etc/security/limits.conf, or just by running ulimit before execing the potentially offending binary) are probably the solution to this.

Your Answer

