
Check her over and let me know if there are any problems ... will do a
full general announce tomorrow for it ...

Fri, 3 Oct 2003 00:39:15 -0300 (ADT)
"Marc G. Fournier" <scrappy@postgresql.org>
Beta4 Tag'd and Bundled ...

Hi,

Many of the regression tests are failing on my OSX 10.2.6 machine. I have
put the regression.diffs file here

http://bugs.sghms.ac.uk/downloads/regression.diffs

Has this been seen before?

Thanks

adam

 
> Check her over and let me know if there are any problems ... will do a
> full general announce tomorrow for it ...
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


Sat, 04 Oct 2003 14:14:56 +0100
Adam Witney <awitney@sghms.ac.uk>
Re: Beta4 Tag'd and Bundled ...

Hi,

Many of the regression tests are failing on my OSX 10.2.6 machine. I have
put the regression.diffs file here

http://bugs.sghms.ac.uk/downloads/regression.diffs

Has this been seen before?

Thanks

adam

 
> Check her over and let me know if there are any problems ... will do a
> full general announce tomorrow for it ...
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


Sat, 04 Oct 2003 14:21:06 +0100
Adam Witney <awitney@sghms.ac.uk>
Re: Beta4 Tag'd and Bundled ...
Adam Witney <awitney@sghms.ac.uk> writes:
> Many of the regression tests are failing on my OSX 10.2.6 machine. I have
> put the regression.diffs file here
> http://bugs.sghms.ac.uk/downloads/regression.diffs

Seems to be all

! psql: FATAL:  sorry, too many clients already

What did initdb set your max_connections to?

			regards, tom lane

Sat, 04 Oct 2003 12:16:45 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Beta4 Tag'd and Bundled ... 
On 4/10/03 5:16 pm, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

> Adam Witney <awitney@sghms.ac.uk> writes:
>> Many of the regression tests are failing on my OSX 10.2.6 machine. I have
>> put the regression.diffs file here
>> http://bugs.sghms.ac.uk/downloads/regression.diffs
> 
> Seems to be all
> 
> ! psql: FATAL:  sorry, too many clients already
> 
> What did initdb set your max_connections to?
> 
> regards, tom lane

>From src/test/regress/log/initdb.log

selecting default max_connections... 10

Is this the info you are asking for?

adam


Sat, 04 Oct 2003 17:30:17 +0100
Adam Witney <awitney@sghms.ac.uk>
Re: Beta4 Tag'd and Bundled ...
Adam Witney <awitney@sghms.ac.uk> writes:
> On 4/10/03 5:16 pm, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
>> What did initdb set your max_connections to?

> From src/test/regress/log/initdb.log
> selecting default max_connections... 10

Hm.  The parallel regression tests require at least 20.  I deliberately
allowed initdb to select values as small as 10 on the theory that
installing and not being able to run the parallel regression tests is
better than not installing at all.  Does anyone want to argue the
opposite?  Perhaps we just need a note in the documentation --- but if
so, where?

			regards, tom lane

Sat, 04 Oct 2003 12:53:42 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Beta4 Tag'd and Bundled ... 
> Hm.  The parallel regression tests require at least 20.  I deliberately
> allowed initdb to select values as small as 10 on the theory that
> installing and not being able to run the parallel regression tests is
> better than not installing at all.  Does anyone want to argue the
> opposite?  Perhaps we just need a note in the documentation --- but if
> so, where?

Another alternative is to have the regression suite discover the max
connections, and defer tests when there are (max_connections - 1)
connections already.
Sat, 04 Oct 2003 13:20:42 -0400
Rod Taylor <rbt@rbt.ca>
Re: Beta4 Tag'd and Bundled ...
Rod Taylor <rbt@rbt.ca> writes:
>> Hm.  The parallel regression tests require at least 20.  I deliberately
>> allowed initdb to select values as small as 10 on the theory that
>> installing and not being able to run the parallel regression tests is
>> better than not installing at all.

> Another alternative is to have the regression suite discover the max
> connections, and defer tests when there are (max_connections - 1)
> connections already.

Maybe.  After thinking a bit more, I seem to recall one of the reasons
for having wide parallel sets in the regression tests is that we
*wanted* to consider inability to support a dozen or two connections as
a serious problem.  If we still believe that old logic, then indeed the
right thing to do is for initdb to insist on setting max_connections no
smaller than 20.  (Pre-7.4, the default setting was generally 32, so
this is still more flexible than before from a portability standpoint.)

			regards, tom lane

Sat, 04 Oct 2003 13:29:23 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Beta4 Tag'd and Bundled ... 
I said:
> Hm.  The parallel regression tests require at least 20.  I deliberately
> allowed initdb to select values as small as 10 on the theory that
> installing and not being able to run the parallel regression tests is
> better than not installing at all.

Actually, after trying to reproduce the problem on my own OS X machine,
I realize that it's a little more subtle than I thought.  The Darwin
port does not use SysV semaphores at all (it uses Posix semaphores) and
so the resource restriction you're hitting must actually be the
max-shared-memory limit, rather than number-of-semaphores as I first
assumed.  max_connections does have an impact on shared memory size,
though not as large as shared_buffers has.

Therefore, the real problem is that initdb initially probes for a
workable number of shared_buffers while using max_connections=5, and
then it selects max_connections while holding shared_buffers constant.
I was thinking that a small max_connections would prevent us from
mistakenly choosing tiny shared_buffers when the system's real
restriction is on number of semaphores.  But what we seem to have here
is that the selected number of buffers was just a little under the
system's max-shared-memory limit, so that max_connections could be
raised to 10 but not to 20.

(BTW, on my OS X machine, with out-of-the-box configuration, initdb
selects shared_buffers 400 and max_connections 20.  I'm guessing that
you had either a nondefault shared memory limit, or some other process
using shared memory.)

Perhaps we should avoid all attempts at cuteness and just run the
initial probes for workable shared_buffers with max_connections=20,
as well as making that be the minimum max_connections value probed for.

Anyone see a better way?

			regards, tom lane

Sat, 04 Oct 2003 15:10:53 -0400
Tom Lane <tgl@sss.pgh.pa.us>
max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled ...)

Tom Lane <tgl@sss.pgh.pa.us> writes:

> (BTW, on my OS X machine, with out-of-the-box configuration, initdb
> selects shared_buffers 400 and max_connections 20.  I'm guessing that
> you had either a nondefault shared memory limit, or some other process
> using shared memory.)

This points out another issue with this approach of probing for the maximum
shared memory. There might be another program using shared memory when the
probe is done -- or worse when the database is started but *not* when the
probe is run.

Perhaps the shared_buffers should only be set to 50% of the maximum size
probed? That would a) give postgres a decent chance of starting even if some
other program uses some amount of the shared memory between initdb and
database starting. and b) leave enough memory for a reasonable
max_connections?

> Anyone see a better way?

Switch everything to mmap and pthreads and dump all this antiquated SysV IPC
and semaphore junk? *DUCK*

-- 
greg


04 Oct 2003 17:08:57 -0400
Greg Stark <gsstark@mit.edu>
Re: max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled
	...)
Greg Stark <gsstark@mit.edu> writes:
> Perhaps the shared_buffers should only be set to 50% of the maximum size
> probed?

I think it's reasonable to expect the DBA to make any adjustments needed
for changes in environment.  Partly this is because I don't see any
defensible way to do otherwise --- your 50% slop figure is without
foundation in terms of what might really be going on --- and partly
because we'd be handicapping ourselves unnecessarily if there *aren't*
any subsequent changes in environment.

On machines where shared memory actually gets used for anything by
default, I think that the default limits are likely to be fairly sane.
If shared memory is tight, then very likely Postgres is the only thing
on the machine that's going to want it.  We might as well use what we
can get.

			regards, tom lane

Sat, 04 Oct 2003 17:25:31 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled
	...)


>>Anyone see a better way?
>>    
>>
>
>Switch everything to mmap and pthreads and dump all this antiquated SysV IPC
>and semaphore junk? *DUCK*
>  
>

You are a brave soult. I salute you.

Sincerely,

Joshua D. Drake



-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC - S/JDBC
Postgresql support, programming, shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL.Org - Editor-N-Chief - http://www.postgresql.org



Sat, 04 Oct 2003 15:00:13 -0700
"Joshua D. Drake" <jd@commandprompt.com>
Re: max_connections/shared_buffers (was Re: Beta4 Tag'd
On 4/10/03 8:10 pm, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:

> I said:
>> Hm.  The parallel regression tests require at least 20.  I deliberately
>> allowed initdb to select values as small as 10 on the theory that
>> installing and not being able to run the parallel regression tests is
>> better than not installing at all.
> 
> Actually, after trying to reproduce the problem on my own OS X machine,
> I realize that it's a little more subtle than I thought.  The Darwin
> port does not use SysV semaphores at all (it uses Posix semaphores) and
> so the resource restriction you're hitting must actually be the
> max-shared-memory limit, rather than number-of-semaphores as I first
> assumed.  max_connections does have an impact on shared memory size,
> though not as large as shared_buffers has.
> 
> Therefore, the real problem is that initdb initially probes for a
> workable number of shared_buffers while using max_connections=5, and
> then it selects max_connections while holding shared_buffers constant.
> I was thinking that a small max_connections would prevent us from
> mistakenly choosing tiny shared_buffers when the system's real
> restriction is on number of semaphores.  But what we seem to have here
> is that the selected number of buffers was just a little under the
> system's max-shared-memory limit, so that max_connections could be
> raised to 10 but not to 20.
> 
> (BTW, on my OS X machine, with out-of-the-box configuration, initdb
> selects shared_buffers 400 and max_connections 20.  I'm guessing that
> you had either a nondefault shared memory limit, or some other process
> using shared memory.)

These are my current settings

    sysctl -w kern.sysv.shmmax=4194304
    sysctl -w kern.sysv.shmmin=1
    sysctl -w kern.sysv.shmmni=32
    sysctl -w kern.sysv.shmseg=8
    sysctl -w kern.sysv.shmall=1024

This is a laptop so I have quite a few other apps running, including my
current PostgreSQL installation (7.2.3), the settings of which are

#max_connections = 32
shared_buffers = 100


Let me know if you need any more info?

Thanks

Adam


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.


Mon, 06 Oct 2003 09:06:11 +0100
Adam Witney <awitney@sghms.ac.uk>
Re: max_connections/shared_buffers (was Re: Beta4 Tag'd	and
Tom Lane writes:

> Perhaps we should avoid all attempts at cuteness and just run the
> initial probes for workable shared_buffers with max_connections=20,
> as well as making that be the minimum max_connections value probed for.
>
> Anyone see a better way?

Maybe just run one loop and try pairs of (shared_buffers, max_connections):

(1000, 100) (800, 50) (600, 40) (400, 30) (200, 20) (50, 10)

-- 
Peter Eisentraut   peter_e@gmx.net


Mon, 6 Oct 2003 19:21:43 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: max_connections/shared_buffers (was Re: Beta4 Tag'd
Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> Anyone see a better way?

> Maybe just run one loop and try pairs of (shared_buffers, max_connections):
> (1000, 100) (800, 50) (600, 40) (400, 30) (200, 20) (50, 10)

Hmm ... that wouldn't work real well as-is, because if max_connections
is being constrained by a limit on SysV semaphores, we could find
ourselves restricting shared_buffers to no purpose.

Would it work to loop as above, and then try a second loop in which we
use the selected max_connections and see if we can make shared_buffers
bigger?

			regards, tom lane

Mon, 06 Oct 2003 13:26:35 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: max_connections/shared_buffers (was Re: Beta4 Tag'd and Bundled
	...)