I did a little more work on the configurable-max-backends patch:

1. initdb didn't work because I had broken bootstrap mode :-(.
Fixed.

2. I separated the hard maximum limit on the number of backends
(MAXBACKENDS, used to size a couple of arrays) from the default
soft limit (now DEF_MAXBACKENDS).

3. The only cost of enlarging MAXBACKENDS is about 32 bytes per
array slot, so I set MAXBACKENDS at 1024 while leaving the
default DEF_MAXBACKENDS at 64.  (I more than bought back the 32K
by reducing the number of allocated spinlocks to what we were
actually using, anyway.)

4. Upshot: default limit on number of backends is 64, same as it
ever was, but you can set it as high as 1024 without a recompile.
Just start the postmaster with desired -N switch.  (Of course you
might need to reconfigure your kernel first ;-).)

5. Allocation of semaphores and shared memory is now based on
-N switch value (default or specified) rather than the MAXBACKENDS
constant.

			regards, tom lane

Sat, 20 Feb 1999 22:47:55 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: Max backend limits cleaned up
> 5. Allocation of semaphores and shared memory is now based on
> -N switch value (default or specified) rather than the MAXBACKENDS
> constant.

sgml and man documenation updates, right?  Or should I do it?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sat, 20 Feb 1999 22:59:12 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up
Bruce Momjian <maillist@candle.pha.pa.us> writes:
>> 5. Allocation of semaphores and shared memory is now based on
>> -N switch value (default or specified) rather than the MAXBACKENDS
>> constant.

> sgml and man documenation updates, right?  Or should I do it?

I put something into the docs last night in the places where configure
and postmaster switches are described.

I am thinking, though, that we also ought to have FAQ entries under
headings like:

	I get "IpcSemaphoreCreate: semget failed (No space left on device)"
	when I try to start the postmaster

	I get 'Sorry, too many clients already' when trying to connect

If you like, I'll try to write up a first cut at these.

			regards, tom lane

Sun, 21 Feb 1999 13:43:47 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up 
> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> >> 5. Allocation of semaphores and shared memory is now based on
> >> -N switch value (default or specified) rather than the MAXBACKENDS
> >> constant.
> 
> > sgml and man documenation updates, right?  Or should I do it?
> 
> I put something into the docs last night in the places where configure
> and postmaster switches are described.
> 
> I am thinking, though, that we also ought to have FAQ entries under
> headings like:
> 
> 	I get "IpcSemaphoreCreate: semget failed (No space left on device)"
> 	when I try to start the postmaster
> 
> 	I get 'Sorry, too many clients already' when trying to connect
> 
> If you like, I'll try to write up a first cut at these.

Sure.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 21 Feb 1999 14:27:36 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up
Yom, I am getting:


(1) cat /u/pg/server.log
IpcSemaphoreCreate: semget failed (No space left on device) key=5432017,
num=16, permission=600

This is without any special switches.  Do I need to do anything?

> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> >> 5. Allocation of semaphores and shared memory is now based on
> >> -N switch value (default or specified) rather than the MAXBACKENDS
> >> constant.
> 
> > sgml and man documenation updates, right?  Or should I do it?
> 
> I put something into the docs last night in the places where configure
> and postmaster switches are described.
> 
> I am thinking, though, that we also ought to have FAQ entries under
> headings like:
> 
> 	I get "IpcSemaphoreCreate: semget failed (No space left on device)"
> 	when I try to start the postmaster
> 
> 	I get 'Sorry, too many clients already' when trying to connect
> 
> If you like, I'll try to write up a first cut at these.
> 
> 			regards, tom lane
> 


-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 21 Feb 1999 17:06:39 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up
I got it working by adding a -N 32 to the postmaster startup.  Looks
like my site BSD/OS can't start 64 backends.  Some of my configuration
is wrong.  Perhaps we need 32 as the default.


> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> >> 5. Allocation of semaphores and shared memory is now based on
> >> -N switch value (default or specified) rather than the MAXBACKENDS
> >> constant.
> 
> > sgml and man documenation updates, right?  Or should I do it?
> 
> I put something into the docs last night in the places where configure
> and postmaster switches are described.
> 
> I am thinking, though, that we also ought to have FAQ entries under
> headings like:
> 
> 	I get "IpcSemaphoreCreate: semget failed (No space left on device)"
> 	when I try to start the postmaster
> 
> 	I get 'Sorry, too many clients already' when trying to connect
> 
> If you like, I'll try to write up a first cut at these.
> 
> 			regards, tom lane
> 


-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 21 Feb 1999 17:24:03 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up
Bruce Momjian <maillist@candle.pha.pa.us> writes:
> I am getting:
> IpcSemaphoreCreate: semget failed (No space left on device) key=5432017,
> num=16, permission=600
> [ later ]
> I got it working by adding a -N 32 to the postmaster startup.  Looks
> like my site BSD/OS can't start 64 backends.  Some of my configuration
> is wrong.  Perhaps we need 32 as the default.

Yeah, I was thinking about that myself.  I left the default -N setting
at 64 on the theory that people who had gone to the trouble of making
sure they had proper kernel configurations should not get surprised by
v6.5 suddenly reducing the default number-of-backends limit.

On the other hand, we have reason to believe that a lot of systems are
not configured to allow Postgres to grab 64 semaphores, so if we don't
reduce the default -N value we will almost certainly see a lot of gripes
just like the above when people move to 6.5.  (I think -N 32 would work
as a default on minimally-configured systems, but cannot prove it.)

I haven't got a real strong feeling either way.  Opinions?

			regards, tom lane

Mon, 22 Feb 1999 10:10:22 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up 
Having recently experienced a similar problem with semaphores and
kernel size, I can say it is an issue.  I feel that documentation
will clear it up either way.  Either you lower the default backend
limit, and document how to raise it along with the associated kernel
variables, or leave it alone and document the appropriate steps to
tuning the kernel to accommodate it and how to lower it if you don't
want to tune the kernel.

DwD

> -----Original Message-----
> From: owner-pgsql-hackers@postgreSQL.org
> [mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Tom Lane
> Sent: Monday, February 22, 1999 10:10 AM
> To: Bruce Momjian
> Cc: pgsql-hackers@postgreSQL.org
> Subject: Re: [HACKERS] Re: Max backend limits cleaned up
>
>
> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> > I am getting:
> > IpcSemaphoreCreate: semget failed (No space left on
> device) key=5432017,
> > num=16, permission=600
> > [ later ]
> > I got it working by adding a -N 32 to the postmaster
> startup.  Looks
> > like my site BSD/OS can't start 64 backends.  Some of
> my configuration
> > is wrong.  Perhaps we need 32 as the default.
>
> Yeah, I was thinking about that myself.  I left the
> default -N setting
> at 64 on the theory that people who had gone to the
> trouble of making
> sure they had proper kernel configurations should not get
> surprised by
> v6.5 suddenly reducing the default number-of-backends limit.
>
> On the other hand, we have reason to believe that a lot
> of systems are
> not configured to allow Postgres to grab 64 semaphores,
> so if we don't
> reduce the default -N value we will almost certainly see
> a lot of gripes
> just like the above when people move to 6.5.  (I think -N
> 32 would work
> as a default on minimally-configured systems, but cannot
> prove it.)
>
> I haven't got a real strong feeling either way.  Opinions?
>
> 			regards, tom lane
>


Mon, 22 Feb 1999 11:07:31 -0500
"Daryl W. Dunbar" <daryl@www.com>
RE: [HACKERS] Re: Max backend limits cleaned up 
> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> > I am getting:
> > IpcSemaphoreCreate: semget failed (No space left on device) key=5432017,
> > num=16, permission=600
> > [ later ]
> > I got it working by adding a -N 32 to the postmaster startup.  Looks
> > like my site BSD/OS can't start 64 backends.  Some of my configuration
> > is wrong.  Perhaps we need 32 as the default.
> 
> Yeah, I was thinking about that myself.  I left the default -N setting
> at 64 on the theory that people who had gone to the trouble of making
> sure they had proper kernel configurations should not get surprised by
> v6.5 suddenly reducing the default number-of-backends limit.

The default was always 32, right, so they should not be suprised if they
don't up the limit to 64.  Now, if they modify that config.h value, they
may be suprised.  Is that the problem?


-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 22 Feb 1999 11:55:00 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up
[Charset iso-8859-1 unsupported, filtering to ASCII...]
> Having recently experienced a similar problem with semaphores and
> kernel size, I can say it is an issue.  I feel that documentation
> will clear it up either way.  Either you lower the default backend
> limit, and document how to raise it along with the associated kernel
> variables, or leave it alone and document the appropriate steps to
> tuning the kernel to accommodate it and how to lower it if you don't
> want to tune the kernel.
> 

The issue for me is that novices who are never going to hit the 32-user
limit should be able to install PostgreSQL with no major changes,
including not even a postmaster flag to lower the limit.

If you need more than 32 connections, you will be able to modify the
kernel.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 22 Feb 1999 11:57:21 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up
The default to-date has been 64.  The reason you don't see much
trouble with it is twofold, 1) Linux has a huge default for
semaphores and shared memory, 2) The old memory model allocated
semaphores in blocks of 16 up to MaxBackedId (which was hard coded
to 64).  I did not run into trouble on untuned Solaris until
postmaster tried to start the 49th backend (semaphores 49-64 when my
kernel defaulted to 60).

The new model allocates semaphores and shared memory at startup,
assuring you won't experience midstream troubles like I did.  It
does, however allocate to the max or -N setting, which most users
will probably never reach.

I'd vote for 32 as the default.

DwD

> -----Original Message-----
> From: Bruce Momjian [mailto:maillist@candle.pha.pa.us]
> Sent: Monday, February 22, 1999 11:57 AM
> To: Daryl W. Dunbar
> Cc: tgl@sss.pgh.pa.us; pgsql-hackers@postgreSQL.org
> Subject: Re: [HACKERS] Re: Max backend limits cleaned up
>
>
> [Charset iso-8859-1 unsupported, filtering to ASCII...]
> > Having recently experienced a similar problem with
> semaphores and
> > kernel size, I can say it is an issue.  I feel that
> documentation
> > will clear it up either way.  Either you lower the
> default backend
> > limit, and document how to raise it along with the
> associated kernel
> > variables, or leave it alone and document the
> appropriate steps to
> > tuning the kernel to accommodate it and how to lower it
> if you don't
> > want to tune the kernel.
> >
>
> The issue for me is that novices who are never going to
> hit the 32-user
> limit should be able to install PostgreSQL with no major changes,
> including not even a postmaster flag to lower the limit.
>
> If you need more than 32 connections, you will be able to
> modify the
> kernel.
>
> --
>   Bruce Momjian                        |
> http://www.op.net/~candle
>   maillist@candle.pha.pa.us
>        |  (610) 853-3000
>   +  If your life is a hard drive,     |  830 Blythe Avenue
>   +  Christ can be your backup.        |  Drexel Hill,
> Pennsylvania 19026
>


Mon, 22 Feb 1999 12:13:10 -0500
"Daryl W. Dunbar" <daryl@www.com>
RE: [HACKERS] Re: Max backend limits cleaned up
Bruce Momjian <maillist@candle.pha.pa.us> writes:
>> Yeah, I was thinking about that myself.  I left the default -N setting
>> at 64 on the theory that people who had gone to the trouble of making
>> sure they had proper kernel configurations should not get surprised by
>> v6.5 suddenly reducing the default number-of-backends limit.

> The default was always 32, right, so they should not be suprised if they
> don't up the limit to 64.

No, the default MaxBackendId was 64 (unless that's been changed
recently?).  A stock installation of 6.4 will support 64 backends
assuming that you have adequate kernel settings for that.

			regards, tom lane

Mon, 22 Feb 1999 12:51:17 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up 
[Charset iso-8859-1 unsupported, filtering to ASCII...]
> The default to-date has been 64.  The reason you don't see much
> trouble with it is twofold, 1) Linux has a huge default for
> semaphores and shared memory, 2) The old memory model allocated
> semaphores in blocks of 16 up to MaxBackedId (which was hard coded
> to 64).  I did not run into trouble on untuned Solaris until
> postmaster tried to start the 49th backend (semaphores 49-64 when my
> kernel defaulted to 60).
> 
> The new model allocates semaphores and shared memory at startup,
> assuring you won't experience midstream troubles like I did.  It
> does, however allocate to the max or -N setting, which most users
> will probably never reach.
> 
> I'd vote for 32 as the default.

I understand now.  The stuff is preallocated.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Mon, 22 Feb 1999 12:54:47 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up
>> I am thinking, though, that we also ought to have FAQ entries under
>> headings like:
>> I get "IpcSemaphoreCreate: semget failed (No space left on device)"
>> when I try to start the postmaster
>> I get 'Sorry, too many clients already' when trying to connect
>> 
>> If you like, I'll try to write up a first cut at these.

> Sure.

Attached are some proposed diffs against the copy of the FAQ that's
currently in CVS ... not sure if that is the master copy or not.

			regards, tom lane

*** FAQ~	Sat Mar  6 17:10:05 1999
--- FAQ	Sat Mar  6 18:00:22 1999
***************
*** 59,66 ****
     2.10) All my servers crash under concurrent table access. Why?
     2.11) How do I tune the database engine for better performance?
     2.12) What debugging features are available in PostgreSQL?
!    2.13) How do I enable more than 64 concurrent backends?
!    2.14) What non-unix ports are available?
     
                             Operational questions
                                        
--- 59,67 ----
     2.10) All my servers crash under concurrent table access. Why?
     2.11) How do I tune the database engine for better performance?
     2.12) What debugging features are available in PostgreSQL?
!    2.13) When I try to start the postmaster, I get IpcSemaphoreCreate errors.
!    2.14) I get 'Sorry, too many clients' when trying to connect.
!    2.15) What non-unix ports are available?
     
                             Operational questions
                                        
***************
*** 384,391 ****
     You either do not have shared memory configured properly in kernel or
     you need to enlarge the shared memory available in the kernel. The
     exact amount you need depends on your architecture and how many
!    buffers you configure postmaster to run with. For most systems, with
!    default buffer sizes, you need a minimum of ~760K.
     
    2.7) I have changed a source file, but a recompile does not see the change?
    
--- 385,393 ----
     You either do not have shared memory configured properly in kernel or
     you need to enlarge the shared memory available in the kernel. The
     exact amount you need depends on your architecture and how many
!    buffers and backend processes you configure postmaster to run with.
!    For most systems, with default numbers of buffers and processes, you
!    need a minimum of ~1MB.
     
    2.7) I have changed a source file, but a recompile does not see the change?
    
***************
*** 420,426 ****
     If you are doing a lot of inserts, consider doing them in a large
     batch using the copy command. This is much faster than single
     individual inserts. Second, statements not in a begin work/commit
!    transaction block are considered to be their in their own transaction.
     Consider performing several statements in a single transaction block.
     This reduces the transaction overhead. Also consider dropping and
     recreating indices when making large data changes.
--- 422,428 ----
     If you are doing a lot of inserts, consider doing them in a large
     batch using the copy command. This is much faster than single
     individual inserts. Second, statements not in a begin work/commit
!    transaction block are considered to be in their own transaction.
     Consider performing several statements in a single transaction block.
     This reduces the transaction overhead. Also consider dropping and
     recreating indices when making large data changes.
***************
*** 482,494 ****
     pgsql/data/base/dbname directory. The client profile file will be put
     in the current directory.
     
!   2.13) How do I enable more than 64 concurrent backends?
!   
!    Edit include/storage/sinvaladt.h, and change the value of
!    MaxBackendId. In the future, we plan to make this a configurable
!    prameter.
     
!   2.14) What non-unix ports are available?
    
     It is possible to compile the libpq C library, psql, and other
     interfaces and binaries to run on MS Windows platforms. In this case,
--- 484,530 ----
     pgsql/data/base/dbname directory. The client profile file will be put
     in the current directory.
     
!    2.13) When I try to start the postmaster, I get IpcSemaphoreCreate errors.
! 
!    If the error message is "IpcSemaphoreCreate: semget failed (No space left
!    on device)" then your kernel is not configured with enough semaphores.
!    Postgres needs one semaphore per potential backend process.  A temporary
!    solution is to start the postmaster with a smaller limit on the number of
!    backend processes (use -N with a parameter less than its default, 32).
!    A more permanent solution is to increase your kernel's SEMMNS and SEMMNI
!    parameters.
! 
!    If the error message is something else, you might not have semaphore
!    support configured in your kernel at all.
! 
!    2.14) I get 'Sorry, too many clients' when trying to connect.
! 
!    You need to increase the postmaster's limit on how many concurrent backend
!    processes it can start.
! 
!    In Postgres 6.5, the default limit is 32 processes.  You can increase it
!    by restarting the postmaster with a suitable -N value.  With the default
!    configuration you can set -N as large as 1024; if you need more, you'll
!    need to increase MAXBACKENDS in include/config.h and rebuild Postgres.
!    You can set the default value of -N at configuration time, if you like,
!    using configure's --with-maxbackends switch.
! 
!    Note that if you make -N larger than 32, you should consider increasing
!    -B beyond its default of 64.  For large numbers of backend processes,
!    you are also likely to find that you need to increase various Unix kernel
!    configuration parameters.  Things to check include the maximum size of
!    shared memory blocks (SHMMAX), the maximum number of semaphores (SEMMNS and
!    SEMMNI), the maximum number of processes (NPROC), the maximum number of
!    processes per user (MAXUPRC), and the maximum number of open files (NFILE
!    and NINODE).  The main reason that Postgres has a limit on the number of
!    allowed backend processes is so that you can ensure that your system
!    won't run out of resources.
! 
!    In Postgres versions prior to 6.5, the maximum number of backends was
!    64, and changing it required a rebuild after altering the MaxBackendId
!    constant in include/storage/sinvaladt.h.
     
!   2.15) What non-unix ports are available?
    
     It is possible to compile the libpq C library, psql, and other
     interfaces and binaries to run on MS Windows platforms. In this case,

Sat, 06 Mar 1999 18:06:39 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up 
> >> I am thinking, though, that we also ought to have FAQ entries under
> >> headings like:
> >> I get "IpcSemaphoreCreate: semget failed (No space left on device)"
> >> when I try to start the postmaster
> >> I get 'Sorry, too many clients already' when trying to connect
> >> 
> >> If you like, I'll try to write up a first cut at these.
> 
> > Sure.
> 
> Attached are some proposed diffs against the copy of the FAQ that's
> currently in CVS ... not sure if that is the master copy or not.

Applied.  The main file is kept as HTML for the web site, while I
convert it to ASCII for the distribution.  Let me know how it looks.  I
did a little html cleanup.  You can see it on the web site.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 7 Mar 1999 06:51:03 -0500 (EST)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] Re: Max backend limits cleaned up