There is a problem which occurs from time to time and which is a bit 
nasty in business environments.
When the shared memory is eaten up by some application such as Apache 
PostgreSQL will refuse to do what it should do because there is no 
memory around. To many people this looks like a problem relatd to 
stability. Also, it influences availability of the database itself.

I was thinking of a solution which might help to get around this problem:
If we had a flag to tell PostgreSQL that XXX Megs of shared memory 
should be preallocated by PostgreSQL. The database would the sure that 
there is always enough memory around. The problem is that PostgreSQL had 
to care more about memory consumption.

Of course, the best solution is to put PostgreSQL on a separate machine 
but many people don't do it so we have to live with memory leaks caused 
by other software (we have just seen a nasty one in mod_perl).

Does it make sense?

	Regards,

		Hans


-- 
Cybertec Geschwinde u Schoenig
Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
Tel: +43/2952/30706; +43/664/233 90 75
www.cybertec.at, www.postgresql.at, kernel.cybertec.at



Wed, 11 Jun 2003 17:40:45 +0200
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at>
Pre-allocation of shared memory ...

We already pre-allocate all shared memory and resources on postmaster
start.

---------------------------------------------------------------------------

Hans-J�rgen Sch�nig wrote:
> There is a problem which occurs from time to time and which is a bit 
> nasty in business environments.
> When the shared memory is eaten up by some application such as Apache 
> PostgreSQL will refuse to do what it should do because there is no 
> memory around. To many people this looks like a problem relatd to 
> stability. Also, it influences availability of the database itself.
> 
> I was thinking of a solution which might help to get around this problem:
> If we had a flag to tell PostgreSQL that XXX Megs of shared memory 
> should be preallocated by PostgreSQL. The database would the sure that 
> there is always enough memory around. The problem is that PostgreSQL had 
> to care more about memory consumption.
> 
> Of course, the best solution is to put PostgreSQL on a separate machine 
> but many people don't do it so we have to live with memory leaks caused 
> by other software (we have just seen a nasty one in mod_perl).
> 
> Does it make sense?
> 
> 	Regards,
> 
> 		Hans
> 
> 
> -- 
> Cybertec Geschwinde u Schoenig
> Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
> Tel: +43/2952/30706; +43/664/233 90 75
> www.cybertec.at, www.postgresql.at, kernel.cybertec.at
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
> 
> http://www.postgresql.org/docs/faqs/FAQ.html
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Wed, 11 Jun 2003 11:51:01 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
Bruce Momjian wrote:
> We already pre-allocate all shared memory and resources on postmaster
> start.


I guess we allocate memory when a backend starts, don't we?
Or do we allocate when the instance starts?

I have two explanations for the following behaviour:

a. a bug
b. not enough shared memory


WARNING:  Message from PostgreSQL backend:
	The Postmaster has informed me that some other backend
	died abnormally and possibly corrupted shared memory.
	I have rolled back the current transaction and am
	going to terminate your database system connection and exit.
	Please reconnect to the database system and repeat your query.
server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
connection to server was lost

The problem is that this only happens with mod_perl and Apache on the 
same machine so I thought it has to do with a known memory leak in 
mod_perl/Apache. I happens after about two weeks (it seems to occur 
regularily).

> Are you suggesting pre-acquiring resources like oracle does? Like you start a 
> database instance, 350MB memory is gone types?
> 
> One thing I love about postgresql is that it does not do any such silly thing. 
> I agree in the case you suggest, it makes sense.
> 
> If at all postgresql goes that way, I would like to see it configurable. I 
> would rather remove an app. from a machine rather than letting it stamp on 
> other apps feet.


Shridhar. Yes, when preallocating some memory it has to be configurable 
(default = off).







-- 
Cybertec Geschwinde u Schoenig
Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
Tel: +43/2952/30706; +43/664/233 90 75
www.cybertec.at, www.postgresql.at, kernel.cybertec.at



Wed, 11 Jun 2003 19:14:06 +0200
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <hs@cybertec.at>
Re: Pre-allocation of shared memory ...
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <hs@cybertec.at> writes:
> I have two explanations for the following behaviour:
> a. a bug
> b. not enough shared memory

> WARNING:  Message from PostgreSQL backend:
> 	The Postmaster has informed me that some other backend
> 	died abnormally and possibly corrupted shared memory.

Is this a Linux machine?  If so, the true explanation is probably (c):
the kernel is kill 9'ing randomly-chosen database processes whenever
it starts to feel low on memory.  I would suggest checking the
postmaster log to determine the signal number the failed backends are
dying with.  The client-side message does not give nearly enough info
to debug such problems.

There is also possibility (d): you have some bad RAM that is located in
an address range that doesn't get used until the machine is under full
load.  But if the backends are dying with signal 9 then I'll take the
kernel-kill theory.

AFAIK the only good way around this problem is to use another OS with a
more rational design for handling low-memory situations.  No other Unix
does anything remotely as brain-dead as what Linux does.  Or bug your
favorite Linux kernel hacker to fix the kernel.

			regards, tom lane

Wed, 11 Jun 2003 16:24:22 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Pre-allocation of shared memory ... 
Tom Lane wrote:
> =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <hs@cybertec.at> writes:
> > I have two explanations for the following behaviour:
> > a. a bug
> > b. not enough shared memory
> 
> > WARNING:  Message from PostgreSQL backend:
> > 	The Postmaster has informed me that some other backend
> > 	died abnormally and possibly corrupted shared memory.
> 
> Is this a Linux machine?  If so, the true explanation is probably (c):
> the kernel is kill 9'ing randomly-chosen database processes whenever
> it starts to feel low on memory.  I would suggest checking the
> postmaster log to determine the signal number the failed backends are
> dying with.  The client-side message does not give nearly enough info
> to debug such problems.
> 
> There is also possibility (d): you have some bad RAM that is located in
> an address range that doesn't get used until the machine is under full
> load.  But if the backends are dying with signal 9 then I'll take the
> kernel-kill theory.
> 
> AFAIK the only good way around this problem is to use another OS with a
> more rational design for handling low-memory situations.  No other Unix
> does anything remotely as brain-dead as what Linux does.  Or bug your
> favorite Linux kernel hacker to fix the kernel.

Is there no sysctl way to disable such kills?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Wed, 11 Jun 2003 16:28:50 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
Bruce Momjian <pgman@candle.pha.pa.us> writes:

> Tom Lane wrote:
> > AFAIK the only good way around this problem is to use another OS with a
> > more rational design for handling low-memory situations.  No other Unix
> > does anything remotely as brain-dead as what Linux does.  Or bug your
> > favorite Linux kernel hacker to fix the kernel.
> 
> Is there no sysctl way to disable such kills?

The -ac kernel patches from Alan Cox have a sysctl to control memory
overcommit--you can set it to track memory usage and fail allocations
when memory runs out, rather than the random kill behavior.  I'm not
sure whether those have made it into the stock kernel yet, but the
vendor kernels (such as Red Hat's) might have it too.

-Doug

11 Jun 2003 19:35:20 -0400
Doug McNaught <doug@mcnaught.org>
Re: Pre-allocation of shared memory ...
On Wed, Jun 11, 2003 at 07:35:20PM -0400, Doug McNaught wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> 
> > Is there no sysctl way to disable such kills?
> 
> The -ac kernel patches from Alan Cox have a sysctl to control memory
> overcommit--you can set it to track memory usage and fail allocations
> when memory runs out, rather than the random kill behavior.  I'm not
> sure whether those have made it into the stock kernel yet, but the
> vendor kernels (such as Red Hat's) might have it too.

Yeah, I see it in the Mandrake kernel.  But it's not in stock 2.4.19, so
you can't assume everybody has it.

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"ï¿½Quï¿½ importan los aï¿½os?  Lo que realmente importa es comprobar que
a fin de cuentas la mejor edad de la vida es estar vivo"  (Mafalda)

Wed, 11 Jun 2003 19:42:56 -0400
Alvaro Herrera <alvherre@dcc.uchile.cl>
Re: Pre-allocation of shared memory ...
 > Yeah, I see it in the Mandrake kernel.  But it's not in stock 2.4.19, so
 > you can't assume everybody has it.
 >

We had this problem on a recent version of good old Slackware.
I think we also had it on RedHat 8 or so.

Doing this kind of killing is definitely a bad habit. I thought it had 
it had to do with something else so my proposal for pre-allocation seems 
to be pretty obsolete ;).

Thanks a lot.

	Hans


-- 
Cybertec Geschwinde u Schoenig
Ludo-Hartmannplatz 1/14, A-1160 Vienna, Austria
Tel: +43/2952/30706; +43/664/233 90 75
www.cybertec.at, www.postgresql.at, kernel.cybertec.at



Thu, 12 Jun 2003 07:34:28 +0200
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <hs@cybertec.at>
Re: Pre-allocation of shared memory ...

On this machine (RH9, kernel 2.4.20-18.9) the docs say (in
/usr/src/linux-2.4/Documentation/vm/overcommit-accounting ):

-----------------
The Linux kernel supports four overcommit handling modes

0       -       Heuristic overcommit handling. Obvious overcommits of
                address space are refused. Used for a typical system. It
                ensures a seriously wild allocation fails while allowing
                overcommit to reduce swap usage

1       -       No overcommit handling. Appropriate for some scientific
                applications

2       -       (NEW) strict overcommit. The total address space commit
                for the system is not permitted to exceed swap + half ram.
                In almost all situations this means a process will not be
                killed while accessing pages but only by malloc failures
                that are reported back by the kernel mmap/brk code.

3       -       (NEW) paranoid overcommit The total address space commit
                for the system is not permitted to exceed swap. The machine
                will never kill a process accessing pages it has mapped
                except due to a bug (ie report it!)
----------------------

So maybe

  sysctl -w vm.overcommit_memory=3

is what's needed? I guess you might pay a performance hit for doing that,
though.

andrew

> > Yeah, I see it in the Mandrake kernel.  But it's not in stock 2.4.19,
> > so you can't assume everybody has it.
> >
>
> We had this problem on a recent version of good old Slackware.
> I think we also had it on RedHat 8 or so.
>
> Doing this kind of killing is definitely a bad habit. I thought it had
> it had to do with something else so my proposal for pre-allocation
> seems  to be pretty obsolete ;).
>
> Thanks a lot.
>
> 	Hans




Thu, 12 Jun 2003 04:35:17 -0500 (EST)
"Andrew Dunstan" <andrew@dunslane.net>
Re: Pre-allocation of shared memory ...

A couple of points:

. It is probably a good idea to put do this via /etc/sysctl.conf, which will
be called earlyish by init scripts (on RH9 it is in the network startup
file, for some reason).

. The setting is not available on all kernel versions AFAIK. The admin needs
to check the docs. I have no idea when this went into the kernel, and no
time to spend finding out. Even if we knew, it might have gone into vendor
kernels at other odd times  - there are often times when the vendors are in
advance of the officially released kernels.

Andrew


Bruce wrote:
>
> OK, new text is:
>
>       <para>
>        Linux has poor default memory overcommit behavior.  Rather than
>        failing if it can not reserve enough memory, it returns success,
>        but later fails when the memory can't be mapped and terminates
>        the application with <literal>kill -9</>.  To prevent
>        unpredictable process termination, use:
> <programlisting>
> sysctl -w vm.overcommit_memory=3
> </programlisting>
>        Note, you will need enough swap space to cover all your memory
>        needs.
>       </para>
>      </listitem>
>     </varlistentry>
>
> ---------------------------------------------------------------------------
>
> Tom Lane wrote:
>> Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> > OK, doc patch attached and applied.  Improvements?
>>
>> I think it would be worth spending another sentence to tell people
>> exactly what the symptom looks like, ie, backends dying with signal 9.
>>
>> 			regards, tom lane
>>
>
> --
>  Bruce Momjian                        |  http://candle.pha.pa.us
>  pgman@candle.pha.pa.us               |  (610) 359-1001
>  +  If your life is a hard drive,     |  13 Roberts Road
>  +  Christ can be your backup.        |  Newtown Square, Pennsylvania
>  19073
>
> ---------------------------(end of
> broadcast)--------------------------- TIP 2: you can get off all lists
> at once with the unregister command
>    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)




Thu, 12 Jun 2003 07:28:47 -0500 (EST)
"Andrew Dunstan" <andrew@dunslane.net>
Re: Pre-allocation of shared memory ...
Tom Lane wrote:
> Is this a Linux machine?  If so, the true explanation is probably (c):
> the kernel is kill 9'ing randomly-chosen database processes whenever
> it starts to feel low on memory.  I would suggest checking the
> postmaster log to determine the signal number the failed backends are
> dying with.  The client-side message does not give nearly enough info
> to debug such problems.
> 
> AFAIK the only good way around this problem is to use another OS with a
> more rational design for handling low-memory situations.  No other Unix
> does anything remotely as brain-dead as what Linux does.  Or bug your
> favorite Linux kernel hacker to fix the kernel.

Tom-

Just curious.  What would a rationally designed OS do in an out of 
memory situation?

It seems like from the discussions I've read about the subject there 
really is no rational solution to this irrational problem.

Some solutions such as "suspend process, write image to file" and 
"increase swap space" assume available disk space, which is obviously 
not guaranteed to be avaliable.

-- 
-**-*-*---*-*---*-*---*-----*-*-----*---*-*---*-----*-----*-*-----*---
  Jon Lapham  <lapham@extracta.com.br>          Rio de Janeiro, Brasil
  Work: Extracta Molï¿½culas Naturais SA     http://www.extracta.com.br/
  Web: http://www.jandr.org/
***-*--*----*-------*------------*--------------------*---------------



Thu, 12 Jun 2003 11:13:43 -0300
Jon Lapham <lapham@extracta.com.br>
Re: Pre-allocation of shared memory ...
Jon Lapham <lapham@extracta.com.br> writes:
> Just curious.  What would a rationally designed OS do in an out of 
> memory situation?

Fail malloc() requests.

The sysctl docs that Andrew Dunstan just provided give some insight into
the problem: the default behavior of Linux is to promise more virtual
memory than it can actually deliver.  That is, it allows malloc to
succeed even when it's not going to be able to actually provide the
address space when push comes to shove.  When called to stand and
deliver, the kernel has no way to report failure (other than perhaps a
software-induced SIGSEGV, which would hardly be an improvement).  So it
kills the process instead.  Unfortunately, the process that happens to
be in the line of fire at this point could be any process, not only the
one that made unreasonable memory demands.

This is perhaps an okay behavior for desktop systems being run by
people who are accustomed to Microsoft-like reliability.  But to make it
the default is brain-dead, and to make it the only available behavior
(as seems to have been true until very recently) defies belief.  The
setting now called "paranoid overcommit" is IMHO the *only* acceptable
one for any sort of server system.  With anything else, you risk having
critical userspace daemons killed through no fault of their own.

			regards, tom lane

Thu, 12 Jun 2003 10:29:31 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Pre-allocation of shared memory ... 
Tom Lane wrote:
 > [snip]
> The
> setting now called "paranoid overcommit" is IMHO the *only* acceptable
> one for any sort of server system.  With anything else, you risk having
> critical userspace daemons killed through no fault of their own.

Wow.  Thanks for the info.  I found the documentation you are referring 
to in Documentation/vm/overcommit-accounting (on a stock RH9 machine).

It seems that the overcommit policy is set via the sysctl 
`vm.overcommit_memory'.  So...

[root@bilbo src]# sysctl -a | grep -i overcommit
vm.overcommit_memory = 0

...the default seems to be "Heuristic overcommit handling".  It seems 
that what we want is "vm.overcommit_memory = 3" for paranoid overcommit.

Thanks for getting to the bottom of this Tom.  It *is* insane that the 
default isn't "paranoid overcommit".

-- 
-**-*-*---*-*---*-*---*-----*-*-----*---*-*---*-----*-----*-*-----*---
  Jon Lapham  <lapham@extracta.com.br>          Rio de Janeiro, Brasil
  Work: Extracta Molï¿½culas Naturais SA     http://www.extracta.com.br/
  Web: http://www.jandr.org/
***-*--*----*-------*------------*--------------------*---------------



Thu, 12 Jun 2003 11:52:46 -0300
Jon Lapham <lapham@extracta.com.br>
Re: Pre-allocation of shared memory ...

What really kills [:-)] me is that they allocate memory assuming I will
not be using it all, then terminate the executable in an unrecoverable
way when I go to use the memory.

And, they make a judgement on users who don't want this by calling them
"paranoid".

I will add something to the docs about this.

---------------------------------------------------------------------------

Tom Lane wrote:
> Jon Lapham <lapham@extracta.com.br> writes:
> > Just curious.  What would a rationally designed OS do in an out of 
> > memory situation?
> 
> Fail malloc() requests.
> 
> The sysctl docs that Andrew Dunstan just provided give some insight into
> the problem: the default behavior of Linux is to promise more virtual
> memory than it can actually deliver.  That is, it allows malloc to
> succeed even when it's not going to be able to actually provide the
> address space when push comes to shove.  When called to stand and
> deliver, the kernel has no way to report failure (other than perhaps a
> software-induced SIGSEGV, which would hardly be an improvement).  So it
> kills the process instead.  Unfortunately, the process that happens to
> be in the line of fire at this point could be any process, not only the
> one that made unreasonable memory demands.
> 
> This is perhaps an okay behavior for desktop systems being run by
> people who are accustomed to Microsoft-like reliability.  But to make it
> the default is brain-dead, and to make it the only available behavior
> (as seems to have been true until very recently) defies belief.  The
> setting now called "paranoid overcommit" is IMHO the *only* acceptable
> one for any sort of server system.  With anything else, you risk having
> critical userspace daemons killed through no fault of their own.
> 
> 			regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
> 
> http://archives.postgresql.org
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 11:05:12 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> What really kills [:-)] me is that they allocate memory assuming I will
> not be using it all, then terminate the executable in an unrecoverable
> way when I go to use the memory.

To be fair, I'm probably misstating things by referring to malloc().
The big problem probably comes from fork() with copy-on-write --- the
kernel has no good way to estimate how much of the shared address space
will eventually become private modified copies, but it can be forgiven
for wanting to make less than the worst-case assumption.

Still, if you are wanting to run a reliable server, I think worst-case
assumption is exactly what you want.  Swap space is cheap, and there's
no reason you shouldn't have enough swap to support the worst-case
situation.  If the swap area goes largely unused, that's fine.

The policy they're calling "paranoid overcommit" (don't allocate more
virtual memory than you have swap) is as far as I know the standard on
all Unixen other than Linux; certainly it's the traditional behavior.

			regards, tom lane

Thu, 12 Jun 2003 11:19:51 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Pre-allocation of shared memory ... 
OK, doc patch attached and applied.  Improvements?

---------------------------------------------------------------------------

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > What really kills [:-)] me is that they allocate memory assuming I will
> > not be using it all, then terminate the executable in an unrecoverable
> > way when I go to use the memory.
> 
> To be fair, I'm probably misstating things by referring to malloc().
> The big problem probably comes from fork() with copy-on-write --- the
> kernel has no good way to estimate how much of the shared address space
> will eventually become private modified copies, but it can be forgiven
> for wanting to make less than the worst-case assumption.
> 
> Still, if you are wanting to run a reliable server, I think worst-case
> assumption is exactly what you want.  Swap space is cheap, and there's
> no reason you shouldn't have enough swap to support the worst-case
> situation.  If the swap area goes largely unused, that's fine.
> 
> The policy they're calling "paranoid overcommit" (don't allocate more
> virtual memory than you have swap) is as far as I know the standard on
> all Unixen other than Linux; certainly it's the traditional behavior.
> 
> 			regards, tom lane
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Index: doc/src/sgml/runtime.sgml
===================================================================
RCS file: /cvsroot/pgsql-server/doc/src/sgml/runtime.sgml,v
retrieving revision 1.184
diff -c -c -r1.184 runtime.sgml
*** doc/src/sgml/runtime.sgml	11 Jun 2003 22:13:21 -0000	1.184
--- doc/src/sgml/runtime.sgml	12 Jun 2003 15:29:45 -0000
***************
*** 2780,2785 ****
--- 2780,2795 ----
          <filename>/usr/src/linux/include/asm-<replaceable>xxx</>/shmpara
          m.h</> and <filename>/usr/src/linux/include/linux/sem.h</>.
         </para>
+ 
+        <para>
+         Linux has poor default memory overcommit behavior.  Rather than
+         failing if it can not reserve enough memory, it returns success, 
+         but later fails when the memory can't be mapped and terminates 
+         the application.  To prevent unpredictable process termination, use:
+ <programlisting>
+ sysctl -w vm.overcommit_memory=3
+ </programlisting>
+        </para>
        </listitem>
       </varlistentry>
Thu, 12 Jun 2003 11:31:28 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> OK, doc patch attached and applied.  Improvements?

I think it would be worth spending another sentence to tell people
exactly what the symptom looks like, ie, backends dying with signal 9.

			regards, tom lane

Thu, 12 Jun 2003 11:39:36 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Pre-allocation of shared memory ... 

I have added the following sentence to the docs too:

        Note, you will need enough swap space to cover all your memory
	needs.

I still wish Linux would just fail the fork/malloc when memory is low,
rather than requiring swap for everything _or_ overcommitting.  I wonder
if making a unified buffer cache just made that too hard to do.

---------------------------------------------------------------------------

Andrew Dunstan wrote:
> 
> On this machine (RH9, kernel 2.4.20-18.9) the docs say (in
> /usr/src/linux-2.4/Documentation/vm/overcommit-accounting ):
> 
> -----------------
> The Linux kernel supports four overcommit handling modes
> 
> 0       -       Heuristic overcommit handling. Obvious overcommits of
>                 address space are refused. Used for a typical system. It
>                 ensures a seriously wild allocation fails while allowing
>                 overcommit to reduce swap usage
> 
> 1       -       No overcommit handling. Appropriate for some scientific
>                 applications
> 
> 2       -       (NEW) strict overcommit. The total address space commit
>                 for the system is not permitted to exceed swap + half ram.
>                 In almost all situations this means a process will not be
>                 killed while accessing pages but only by malloc failures
>                 that are reported back by the kernel mmap/brk code.
> 
> 3       -       (NEW) paranoid overcommit The total address space commit
>                 for the system is not permitted to exceed swap. The machine
>                 will never kill a process accessing pages it has mapped
>                 except due to a bug (ie report it!)
> ----------------------
> 
> So maybe
> 
>   sysctl -w vm.overcommit_memory=3
> 
> is what's needed? I guess you might pay a performance hit for doing that,
> though.
> 
> andrew
> 
> > > Yeah, I see it in the Mandrake kernel.  But it's not in stock 2.4.19,
> > > so you can't assume everybody has it.
> > >
> >
> > We had this problem on a recent version of good old Slackware.
> > I think we also had it on RedHat 8 or so.
> >
> > Doing this kind of killing is definitely a bad habit. I thought it had
> > it had to do with something else so my proposal for pre-allocation
> > seems  to be pretty obsolete ;).
> >
> > Thanks a lot.
> >
> > 	Hans
> 
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 11:56:57 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...

OK, new text is:

       <para>
        Linux has poor default memory overcommit behavior.  Rather than
        failing if it can not reserve enough memory, it returns success,
        but later fails when the memory can't be mapped and terminates
        the application with <literal>kill -9</>.  To prevent unpredictable
        process termination, use:
<programlisting>
sysctl -w vm.overcommit_memory=3
</programlisting>
        Note, you will need enough swap space to cover all your memory needs.
       </para>
      </listitem>
     </varlistentry>

---------------------------------------------------------------------------

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > OK, doc patch attached and applied.  Improvements?
> 
> I think it would be worth spending another sentence to tell people
> exactly what the symptom looks like, ie, backends dying with signal 9.
> 
> 			regards, tom lane
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 11:58:44 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...

Well, let's see what feedback we get.

---------------------------------------------------------------------------

Andrew Dunstan wrote:
> 
> A couple of points:
> 
> . It is probably a good idea to put do this via /etc/sysctl.conf, which will
> be called earlyish by init scripts (on RH9 it is in the network startup
> file, for some reason).
> 
> . The setting is not available on all kernel versions AFAIK. The admin needs
> to check the docs. I have no idea when this went into the kernel, and no
> time to spend finding out. Even if we knew, it might have gone into vendor
> kernels at other odd times  - there are often times when the vendors are in
> advance of the officially released kernels.
> 
> Andrew
> 
> 
> Bruce wrote:
> >
> > OK, new text is:
> >
> >       <para>
> >        Linux has poor default memory overcommit behavior.  Rather than
> >        failing if it can not reserve enough memory, it returns success,
> >        but later fails when the memory can't be mapped and terminates
> >        the application with <literal>kill -9</>.  To prevent
> >        unpredictable process termination, use:
> > <programlisting>
> > sysctl -w vm.overcommit_memory=3
> > </programlisting>
> >        Note, you will need enough swap space to cover all your memory
> >        needs.
> >       </para>
> >      </listitem>
> >     </varlistentry>
> >
> > ---------------------------------------------------------------------------
> >
> > Tom Lane wrote:
> >> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> > OK, doc patch attached and applied.  Improvements?
> >>
> >> I think it would be worth spending another sentence to tell people
> >> exactly what the symptom looks like, ie, backends dying with signal 9.
> >>
> >> 			regards, tom lane
> >>
> >
> > --
> >  Bruce Momjian                        |  http://candle.pha.pa.us
> >  pgman@candle.pha.pa.us               |  (610) 359-1001
> >  +  If your life is a hard drive,     |  13 Roberts Road
> >  +  Christ can be your backup.        |  Newtown Square, Pennsylvania
> >  19073
> >
> > ---------------------------(end of
> > broadcast)--------------------------- TIP 2: you can get off all lists
> > at once with the unregister command
> >    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> 
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 12:46:39 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
Tom Lane <tgl@sss.pgh.pa.us> writes:

> The policy they're calling "paranoid overcommit" (don't allocate more
> virtual memory than you have swap) is as far as I know the standard on
> all Unixen other than Linux; certainly it's the traditional behavior.

Uhm, it's traditional for Unixen without extensive shared memory usage like
SunOS 4. But it's not nearly as standard as you say. 

In fact Linux wasn't the first major Unix to behave this way at all. As far as
I know, that honour belongs to AIX. Not coincidentally, one of the first
Unixen to have shared libraries. Hence the AIX invention of SIGDANGER which
told a process its death was imminent.

On AIX the heuristic was to kill the largest process in order to clear up the
most memory -- which had a nasty habit of picking the X server to kill, which
of course, well, it cleared up lots of memory... I think they "fixed" that by
changing the heuristic to kill the *second* biggest process.

I think you'll find this overcommit issue affects many if not most Unixen.
There's a bit of a vicious circle here, a lot of software now have the habit
of starting off by mallocing huge chunks of memory that they never need
because "well the machine has virtual memory so it doesn't cost anything".

-- 
greg


12 Jun 2003 14:41:34 -0400
Greg Stark <gsstark@mit.edu>
Re: Pre-allocation of shared memory ...
Greg Stark <gsstark@mit.edu> writes:
> I think you'll find this overcommit issue affects many if not most Unixen.

I'm unconvinced, because I've only ever heard of the problem affecting
Postgres on Linux.

			regards, tom lane

Thu, 12 Jun 2003 18:19:19 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Pre-allocation of shared memory ... 
Tom Lane wrote:
> Greg Stark <gsstark@mit.edu> writes:
> > I think you'll find this overcommit issue affects many if not most Unixen.
> 
> I'm unconvinced, because I've only ever heard of the problem affecting
> Postgres on Linux.

What I don't understand is why they just don't start failing on
fork/malloc rather than killing things.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 20:08:28 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
On Thu, Jun 12, 2003 at 08:08:28PM -0400, Bruce Momjian wrote:
> > 
> > I'm unconvinced, because I've only ever heard of the problem affecting
> > Postgres on Linux.
> 
> What I don't understand is why they just don't start failing on
> fork/malloc rather than killing things.

I may be way off the mark here, falling into the middle of this as I am,
but it may be because the kernel overcommits the memory (which is sort of
logical in a way given the way fork() works).  That may mean that malloc()
thinks it gets more memory and returns a pointer, but the kernel hasn't
actually committed that address space yet and waits to see if it's ever
going to be needed.

Given the right allocation proportions, this may mean that in the end the
kernel has no way to handle a shortage gracefully by causing fork() or
allocations to fail.  I would assume it then goes through its alternatives
like scaling back its file cache--which it'd probably start to do before
a lot of swapping was needed, so not much to scrape out of that barrel.

After that, where do you go?  Try to find a reasonable process to shoot
in the head.  From what I heard, although I haven't kept current, a lot
of work went into selecting a "reasonable" process, so there will be some
determinism.  And if you have occasion to find out in the first place,
"some determinism" usually means "suspiciously bad luck."


Jeroen


Fri, 13 Jun 2003 02:17:50 +0200
"Jeroen T. Vermeulen" <jtv@xs4all.nl>
Re: Pre-allocation of shared memory ...
I'm not saying you're wrong, but I also think it's true that typical Linux
usage patterns are rather different from those of other *nixen. Linux
started out being able to do a lot with a little, and is still often used
that way - with more functions crammed into boxes with less resources. When
I last worked in a data centre (a few years ago now, for one of the world's
largest companies) they had hundreds of AIX and HP-UX boxes, each well
resourced and each dedicated to exactly one function. I rarely see Linux
being used that way, and I often see it configured with lowish memory and
not nearly enough swap.

In any case, it seems to me we need to have someone check that setting the
vm.overcommit_memory to paranoid will actually stop the postmaster being
killed. I'd love to help but I'm up to my ears in stuff right now. If we
know that we can save the philosophical stuff for another day :-)

cheers

andrew

----- Original Message ----- 
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Greg Stark" <gsstark@mit.edu>
Cc: <pgsql-hackers@postgresql.org>
Sent: Thursday, June 12, 2003 6:19 PM
Subject: Re: [HACKERS] Pre-allocation of shared memory ...


> Greg Stark <gsstark@mit.edu> writes:
> > I think you'll find this overcommit issue affects many if not most
Unixen.
>
> I'm unconvinced, because I've only ever heard of the problem affecting
> Postgres on Linux.
>
> regards, tom lane


Thu, 12 Jun 2003 21:07:12 -0400
"Andrew Dunstan" <andrew@dunslane.net>
Re: Pre-allocation of shared memory ... 
"Jeroen T. Vermeulen" <jtv@xs4all.nl> writes:
> Given the right allocation proportions, this may mean that in the end the
> kernel has no way to handle a shortage gracefully by causing fork() or
> allocations to fail.

Sure it does.  All you need is a conservative allocation policy: fork()
fails if it cannot reserve enough swap space to guarantee that the new
process could write over its entire address space.  Copy-on-write is
an optimization that reduces physical RAM usage, not virtual address
space or swap-space requirements.

Given that swap space is cheap, and that killing random processes is
obviously bad, it's not apparent to me why people think this is not
a good approach --- at least for high-reliability servers.  And Linux
would definitely like to think of itself as a server-grade OS.

> After that, where do you go?  Try to find a reasonable process to shoot
> in the head.  From what I heard, although I haven't kept current, a lot
> of work went into selecting a "reasonable" process, so there will be some
> determinism.

Considering the frequency with which we hear of database backends
getting shot in the head, I'd say those heuristics need lots of work
yet.  I'll take a non-heuristic solution for any system I have to
administer, thanks.

			regards, tom lane

Thu, 12 Jun 2003 21:18:33 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Pre-allocation of shared memory ... 
On Thu, Jun 12, 2003 at 09:18:33PM -0400, Tom Lane wrote:

> Given that swap space is cheap, and that killing random processes is
> obviously bad, it's not apparent to me why people think this is not
> a good approach --- at least for high-reliability servers.  And Linux
> would definitely like to think of itself as a server-grade OS.

Well, it was a toy OS when conceived, that's for sure.  But it's getting
better.

> Considering the frequency with which we hear of database backends
> getting shot in the head, I'd say those heuristics need lots of work
> yet.

Previous versions were said to attempt to kill init.  You have to admit
there has been some progress.

But then there's the problem of people running database servers on
misconfigured machines.  They should know better than not setting enough
swap space, IMHO anyway.

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
Y una voz del caos me hablo y me dijo
"Sonrie y se feliz, podria ser peor".
Y sonrei. Y fui feliz.
Y fue peor.

Thu, 12 Jun 2003 21:33:01 -0400
Alvaro Herrera <alvherre@dcc.uchile.cl>
Re: Pre-allocation of shared memory ...
Tom Lane wrote:
> "Jeroen T. Vermeulen" <jtv@xs4all.nl> writes:
> > Given the right allocation proportions, this may mean that in the end the
> > kernel has no way to handle a shortage gracefully by causing fork() or
> > allocations to fail.
> 
> Sure it does.  All you need is a conservative allocation policy: fork()
> fails if it cannot reserve enough swap space to guarantee that the new
> process could write over its entire address space.  Copy-on-write is
> an optimization that reduces physical RAM usage, not virtual address
> space or swap-space requirements.
> 
> Given that swap space is cheap, and that killing random processes is
> obviously bad, it's not apparent to me why people think this is not
> a good approach --- at least for high-reliability servers.  And Linux
> would definitely like to think of itself as a server-grade OS.

BSD used to require full swap behind all RAM.  I am not sure if that was
changed in BSD 4.4 or in later BSD/OS releases, but it is no longer
true.  I think now it can use RAM or swap as reserved backing store for
fork page modifications.  However, when the system runs of of swap, it
hangs!

> > After that, where do you go?  Try to find a reasonable process to shoot
> > in the head.  From what I heard, although I haven't kept current, a lot
> > of work went into selecting a "reasonable" process, so there will be some
> > determinism.
> 
> Considering the frequency with which we hear of database backends
> getting shot in the head, I'd say those heuristics need lots of work
> yet.  I'll take a non-heuristic solution for any system I have to
> administer, thanks.

You have to love that swap + 1/2 ram option --- when you need four
possible options, there is something wrong with your approach.  :-)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 21:48:51 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> You have to love that swap + 1/2 ram option --- when you need four
> possible options, there is something wrong with your approach.  :-)

I'm still wondering what the "no overcommit handling" option does,
exactly.

			regards, tom lane

Thu, 12 Jun 2003 21:59:23 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Pre-allocation of shared memory ... 
Alvaro Herrera <alvherre@dcc.uchile.cl> writes:

> On Thu, Jun 12, 2003 at 09:18:33PM -0400, Tom Lane wrote:
> 
> > Given that swap space is cheap, and that killing random processes is
> > obviously bad, it's not apparent to me why people think this is not
> > a good approach --- at least for high-reliability servers.  And Linux
> > would definitely like to think of itself as a server-grade OS.

Consider the case of huge processes trying to fork/exec to run ls. It might
seem kind of strange to be getting "Out of memory" errors from your java or
database engine when there are hundreds of megs free on the machine...

I suspect this was less of an issue in the days before copy on write because
vfork was more widely used/implemented. I'm not sure linux even implements
vfork other than just as a wrapper around fork. Even BSD ditched it a while
back though I think I saw that NetBSD reimplemented it since then.

> But then there's the problem of people running database servers on
> misconfigured machines.  They should know better than not setting enough
> swap space, IMHO anyway.

Well, I've seen DBAs say "Since I don't want the database swapping anyways,
I'll make really sure it doesn't swap by just not giving it any swap space --
that's why we bought so much RAM in the first place". It's not obvious that
you need swap to back memory the machine doesn't even report as being in
use...

-- 
greg


12 Jun 2003 22:05:38 -0400
Greg Stark <gsstark@mit.edu>
Re: Pre-allocation of shared memory ...
Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > You have to love that swap + 1/2 ram option --- when you need four
> > possible options, there is something wrong with your approach.  :-)
> 
> I'm still wondering what the "no overcommit handling" option does,
> exactly.

I assume it does no kills, and allows you to commit until you run of of
swap and hang.  This might be the BSD 4.4 behavior, actually.

It is bad to hang the system, but if it reports swap failure, at least
the admin knows why it failed, rather than killing random processes.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 22:10:02 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
Greg Stark wrote:
> I suspect this was less of an issue in the days before copy on write because
> vfork was more widely used/implemented. I'm not sure linux even implements
> vfork other than just as a wrapper around fork. Even BSD ditched it a while
> back though I think I saw that NetBSD reimplemented it since then.
> 
> > But then there's the problem of people running database servers on
> > misconfigured machines.  They should know better than not setting enough
> > swap space, IMHO anyway.
> 
> Well, I've seen DBAs say "Since I don't want the database swapping anyways,
> I'll make really sure it doesn't swap by just not giving it any swap space --
> that's why we bought so much RAM in the first place". It's not obvious that
> you need swap to back memory the machine doesn't even report as being in
> use...

I see no reason RAM can't be used as backing store for possible
copy-on-write use.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Jun 2003 22:11:39 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...

Jeroen T. Vermeulen wrote:
>
>After that, where do you go?  Try to find a reasonable process to shoot
>in the head.  From what I heard, although I haven't kept current, a lot
>of work went into selecting a "reasonable" process, so there will be some
>determinism.

FWIW, you can browse the logic linux uses to choose 
which process to kill here:
  http://lxr.linux.no/source/mm/oom_kill.c

If I read that right, this calculates "points" for each process, where:
   points = vm_size_of_process 
            / sqrt(cpu_time_it_ran)
            / sqrt(sqrt(clock_time_it_had)
            * 2 if the process was niced
            / 4 if the process ran a root
            / 4 if the process had hardware access.
and whichever process has the most points dies.

I'm guessing any database backend (postgres, oracle)
that wasn't part of a long-lived connection seems like 
an especially attractive target to this algorithm.  

(Though hopefully it's all moot now that Andrew / Tom
 found/recommended the paranoid overcommit option, which
 sure seems like the most sane thing for a server to me)

   Ron

PS: Oracle DBAs suffer from the same pain. 
  http://www.cs.helsinki.fi/linux/linux-kernel/2001-12/0098.html
  http://www.ussg.iu.edu/hypermail/linux/kernel/0103.3/0094.html



Thu, 12 Jun 2003 19:22:14 -0700
"Ron Mayer" <ron@intervideo.com>
Re: Pre-allocation of shared memory ...
Bruce Momjian <pgman@candle.pha.pa.us> writes:

> I see no reason RAM can't be used as backing store for possible
> copy-on-write use.

Depends on the scenario. For a database like postgres it would work fairly
well since that RAM is still available for filesystem buffers. For Oracle it
would suck because it's not available for Oracle to allocate to use for its
own buffers. And for a web server with an architecture like Apache it would
suck because it would mean being restricted to a much lower number of
processes than the machine could really handle.

> > I'm still wondering what the "no overcommit handling" option does,
> > exactly.
> 
> I assume it does no kills, and allows you to commit until you run of of
> swap and hang.  This might be the BSD 4.4 behavior, actually.

I think it just makes fork/mmap/sbrk return an error if you run out of swap.
That makes the error appear most likely as malloc() returning null which most
applications don't handle anyways and the user sees the same behaviour:
programs crashing randomly.

Of course that's not what high availability server software does but since
most users' big memory consumers these days seem to be their window manager
and its 3d animated window decorations...

-- 
greg


12 Jun 2003 22:27:00 -0400
Greg Stark <gsstark@mit.edu>
Re: Pre-allocation of shared memory ...
On Thu, Jun 12, 2003 at 07:22:14PM -0700, Ron Mayer wrote:

> FWIW, you can browse the logic linux uses to choose 
> which process to kill here:
>   http://lxr.linux.no/source/mm/oom_kill.c

Hey, this LXR thing is cool.  It'd be nice to have one of those for
Postgres.

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"La naturaleza, tan fragil, tan expuesta a la muerte... y tan viva"

Thu, 12 Jun 2003 23:18:16 -0400
Alvaro Herrera <alvherre@dcc.uchile.cl>
Re: Pre-allocation of shared memory ...
On 12 Jun 2003 at 11:31, Bruce Momjian wrote:

> 
> OK, doc patch attached and applied.  Improvements?

Can we point people to /usr/src/linux/doc...place where they can find more 
documentation  and if their kernel supports it or not.

Bye
 Shridhar

--
Zall's Laws:	(1) Any time you get a mouthful of hot soup, the next thing you do	
   will be wrong.	(2) How long a minute is, depends on which side of the 
bathroom	   door you're on.


Fri, 13 Jun 2003 12:03:20 +0530
"Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
Re: Pre-allocation of shared memory ...
On Thu, Jun 12, 2003 at 07:22:14PM -0700, Ron Mayer wrote:
 
> I'm guessing any database backend (postgres, oracle)
> that wasn't part of a long-lived connection seems like 
> an especially attractive target to this algorithm.  

Yeah, IIRC it tries to pick daemons that can be restarted, or will be
restarted automatically, but may need a lot less memory after that.


Jeroen


Fri, 13 Jun 2003 10:30:49 +0200
"Jeroen T. Vermeulen" <jtv@xs4all.nl>
Re: Pre-allocation of shared memory ...
Shridhar Daithankar wrote:
> On 12 Jun 2003 at 11:31, Bruce Momjian wrote:
> 
> > 
> > OK, doc patch attached and applied.  Improvements?
> 
> Can we point people to /usr/src/linux/doc...place where they can find more 
> documentation  and if their kernel supports it or not.

Yes, we could, but the name of the parameter seems enough.  They
certainly can look that up.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fri, 13 Jun 2003 08:39:02 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
On Thu, Jun 12, 2003 at 10:10:02PM -0400, Bruce Momjian wrote:
> Tom Lane wrote:
> > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > You have to love that swap + 1/2 ram option --- when you need four
> > > possible options, there is something wrong with your approach.  :-)
> > 
> > I'm still wondering what the "no overcommit handling" option does,
> > exactly.
> 
> I assume it does no kills, and allows you to commit until you run of of
> swap and hang.  This might be the BSD 4.4 behavior, actually.

? I thought the idea of no overcommit was that your malloc fails ENOMEM
if there isn't enough memory free for your whole request, rather than
gambling that other processes aren't actually using all of theirs right now
and have pages swapped out. I don't see where the hang comes in..

> It is bad to hang the system, but if it reports swap failure, at least
> the admin knows why it failed, rather than killing random processes.

Yes!

Patrick

Fri, 13 Jun 2003 14:20:21 +0100
Patrick Welche <prlw1@newn.cam.ac.uk>
Re: Pre-allocation of shared memory ...
Patrick Welche wrote:
> On Thu, Jun 12, 2003 at 10:10:02PM -0400, Bruce Momjian wrote:
> > Tom Lane wrote:
> > > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > > You have to love that swap + 1/2 ram option --- when you need four
> > > > possible options, there is something wrong with your approach.  :-)
> > > 
> > > I'm still wondering what the "no overcommit handling" option does,
> > > exactly.
> > 
> > I assume it does no kills, and allows you to commit until you run of of
> > swap and hang.  This might be the BSD 4.4 behavior, actually.
> 
> ? I thought the idea of no overcommit was that your malloc fails ENOMEM
> if there isn't enough memory free for your whole request, rather than
> gambling that other processes aren't actually using all of theirs right now
> and have pages swapped out. I don't see where the hang comes in..

I think there are two important memory cases:

malloc() - should fail right away if it can't reserve the requested
memory;  assuming application request memory they don't use just seems
dumb --- fix the bad apps.

fork() - this is the tricky one because you don't know at fork time who
is going to be sharing the data pages as read-only or doing an exec to
overlay a new process, and who is going to be modifying them and need a
private copy.

I think only the fork case is tricky.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fri, 13 Jun 2003 09:25:49 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Pre-allocation of shared memory ...
On Fri, Jun 13, 2003 at 09:25:49AM -0400, Bruce Momjian wrote:
> 
> malloc() - should fail right away if it can't reserve the requested
> memory;  assuming application request memory they don't use just seems
> dumb --- fix the bad apps.
> 
> fork() - this is the tricky one because you don't know at fork time who
> is going to be sharing the data pages as read-only or doing an exec to
> overlay a new process, and who is going to be modifying them and need a
> private copy.
> 
> I think only the fork case is tricky.

But how do you tell that a malloc() can't get enough memory, once you've
had to overcommit on fork()s?  If a really large program did a regular
fork()/exec() and there wasn't enough free virtual memory to support
the full fork() "just in case the program isn't going to exec()," then
*any* malloc() occurring between the two calls would have to fail.  That
may be better than random killing in theory, but the practical effect
would be close to that.

There's other complications as well, I'm sure.  If this were easy, we
probably wouldn't be discussing this problem now.


Jeroen


Fri, 13 Jun 2003 15:36:00 +0200
"Jeroen T. Vermeulen" <jtv@xs4all.nl>
Re: Pre-allocation of shared memory ...
On Thu, Jun 12, 2003 at 10:10:02PM -0400, Bruce Momjian wrote:
> Tom Lane wrote:
> It is bad to hang the system, but if it reports swap failure, at least
> the admin knows why it failed, rather than killing random processes.
 
I wonder if it might be better to suspend whatever process is trying to
allocate/write to too much memory. At least then you have some chance of
keeping the system up (obviously you'd need to leave some amount free so
you could login to the box to fix things).
-- 
Jim C. Nasby (aka Decibel!)                    jim@nasby.net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Mon, 16 Jun 2003 11:13:18 -0500
"Jim C. Nasby" <jim@nasby.net>
Re: Pre-allocation of shared memory ...