After reading a couple more complaints of hashtable-overflow error
messages, I went ahead and rewrote the hash join modules so that they
don't use fixed-size hash buckets and a fixed-size overflow area.
Instead, each bucket is just a linked list of tuples (thus no wasted
space for underused buckets) and everything is put into a private portal
so that reclaiming the space is easy/quick.  The code is noticeably
shorter and more readable than before.

The limited amount of testing I've been able to do here shows no
problems.

Now: do I commit it, or wait till after 6.5?  I promised Marc the latter
a couple weeks ago, but I am mighty tempted to just go for it...

			regards, tom lane

Tue, 18 May 1999 10:56:25 -0400
Tom Lane <tgl@sss.pgh.pa.us>
I've got it, now should I commit it?
> After reading a couple more complaints of hashtable-overflow error
> messages, I went ahead and rewrote the hash join modules so that they
> don't use fixed-size hash buckets and a fixed-size overflow area.
> Instead, each bucket is just a linked list of tuples (thus no wasted
> space for underused buckets) and everything is put into a private portal
> so that reclaiming the space is easy/quick.  The code is noticeably
> shorter and more readable than before.
> 
> The limited amount of testing I've been able to do here shows no
> problems.
> 
> Now: do I commit it, or wait till after 6.5?  I promised Marc the latter
> a couple weeks ago, but I am mighty tempted to just go for it...

Shhh.  He will never know.  Did you promise Marc, or did you answer him
evasively, like I suggested?

Basically, with the new optimizer, this may be a bug fix because of the
more frequent hashjoins.  That has always been my smokescreen to add the
feature.

:-)

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 18 May 1999 11:18:16 -0400 (EDT)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] I've got it, now should I commit it?
On Tue, 18 May 1999, Bruce Momjian wrote:

> > After reading a couple more complaints of hashtable-overflow error
> > messages, I went ahead and rewrote the hash join modules so that they
> > don't use fixed-size hash buckets and a fixed-size overflow area.
> > Instead, each bucket is just a linked list of tuples (thus no wasted
> > space for underused buckets) and everything is put into a private portal
> > so that reclaiming the space is easy/quick.  The code is noticeably
> > shorter and more readable than before.
> > 
> > The limited amount of testing I've been able to do here shows no
> > problems.
> > 
> > Now: do I commit it, or wait till after 6.5?  I promised Marc the latter
> > a couple weeks ago, but I am mighty tempted to just go for it...
> 
> Shhh.  He will never know.  Did you promise Marc, or did you answer him
> evasively, like I suggested?
> 
> Basically, with the new optimizer, this may be a bug fix because of the
> more frequent hashjoins.  That has always been my smokescreen to add the
> feature.

Tom...make you a deal.  If you are confident enough with the code that
when v6.5 goes out in ~13days, it won't generate more bug reports then its
fixing...go for it. :)
 
Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 


Tue, 18 May 1999 14:32:06 -0300 (ADT)
The Hermit Hacker <scrappy@hub.org>
Re: [HACKERS] I've got it, now should I commit it?
> > Basically, with the new optimizer, this may be a bug fix because of the
> > more frequent hashjoins.  That has always been my smokescreen to add the
> > feature.
> 
> Tom...make you a deal.  If you are confident enough with the code that
> when v6.5 goes out in ~13days, it won't generate more bug reports then its
> fixing...go for it. :)
>  

Ah, man, that was way too easy.  :-)

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 18 May 1999 13:35:58 -0400 (EDT)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] I've got it, now should I commit it?
The Hermit Hacker <scrappy@hub.org> writes:
>> Basically, with the new optimizer, this may be a bug fix because of the
>> more frequent hashjoins.  That has always been my smokescreen to add the
>> feature.

> Tom...make you a deal.  If you are confident enough with the code that
> when v6.5 goes out in ~13days, it won't generate more bug reports then its
> fixing...go for it. :)

OK, you're on --- I feel pretty good about this code, although I'm never
prepared to guarantee zero bugs ;-).  If there are any, we can hope
they'll show up before the end of beta.

A note for anyone testing the new code: the hashtable size (which is now
a target estimate, not a hard limit) is now driven by the postmaster's
-S switch, not the -B switch.  -S seems more reasonable since the table
is private memory in a backend, not shared memory.

			regards, tom lane

Tue, 18 May 1999 17:46:10 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] I've got it, now should I commit it? 
> The Hermit Hacker <scrappy@hub.org> writes:
> >> Basically, with the new optimizer, this may be a bug fix because of the
> >> more frequent hashjoins.  That has always been my smokescreen to add the
> >> feature.
> 
> > Tom...make you a deal.  If you are confident enough with the code that
> > when v6.5 goes out in ~13days, it won't generate more bug reports then its
> > fixing...go for it. :)
> 
> OK, you're on --- I feel pretty good about this code, although I'm never
> prepared to guarantee zero bugs ;-).  If there are any, we can hope
> they'll show up before the end of beta.
> 
> A note for anyone testing the new code: the hashtable size (which is now
> a target estimate, not a hard limit) is now driven by the postmaster's
> -S switch, not the -B switch.  -S seems more reasonable since the table
> is private memory in a backend, not shared memory.

I see no documenation that -B was ever used for hash size.  I see -B for
shared buffers for both postmaster and postgres manual pages.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Tue, 18 May 1999 18:14:32 -0400 (EDT)
Bruce Momjian <maillist@candle.pha.pa.us>
Re: [HACKERS] I've got it, now should I commit it?
Bruce Momjian <maillist@candle.pha.pa.us> writes:
>> A note for anyone testing the new code: the hashtable size (which is now
>> a target estimate, not a hard limit) is now driven by the postmaster's
>> -S switch, not the -B switch.

> I see no documenation that -B was ever used for hash size.

Er, did I say anything about documentation?

The code *was* using NBuffers to size the hashtable, whether or not
that was ever documented anywhere except in the "hash table out of
memory. Use -B parameter to increase buffers" message.  Now it uses
the SortMem variable.

I do have it on my to-do list to update the relevant documentation.
(Yo, Thomas: what's the deadline for 6.5 doco changes?  I've got a bunch
of doc to-dos that I suspect I'd better get moving on...)

			regards, tom lane

Wed, 19 May 1999 00:06:54 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] I've got it, now should I commit it? 
> (Yo, Thomas: what's the deadline for 6.5 doco changes?  I've got a bunch
> of doc to-dos that I suspect I'd better get moving on...)

Yup. Nominally, I should have frozen about May 15, but I still have
some writing I want to do. I can freeze docs in stages; which docs are
you planning on touching?

Bruce, can we get the sgml version of release notes started?

Vadim, you had mentioned some docs for MVCC; where would that show up?
If nothing else, we should update ref/lock.sgml and ref/set.sgml to
cover the grammar changes. And it would be great to have some words
for the User's Guide.

                         - Thomas

-- 
Thomas Lockhart				lockhart@alumni.caltech.edu
South Pasadena, California

Wed, 19 May 1999 15:17:25 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Last call for docs
Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
> I can freeze docs in stages; which docs are
> you planning on touching?

Several, but for most of them I have only small changes.  I will try to
do those tonight so that as much as possible can be frozen, and then let
you know what I still have to work on.

Is there any equivalent in the SGML docs to the postmaster.1 and
postgres.1 man pages (specifically, doco for the postmaster/backend
command line switches)?  I couldn't find it but maybe it's there...

			regards, tom lane

Wed, 19 May 1999 18:58:51 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Last call for docs 
> Several, but for most of them I have only small changes.  I will try to
> do those tonight so that as much as possible can be frozen, and then let
> you know what I still have to work on.

Great.

> Is there any equivalent in the SGML docs to the postmaster.1 and
> postgres.1 man pages (specifically, doco for the postmaster/backend
> command line switches)?  I couldn't find it but maybe it's there...

No, but I want to add them by converting the man pages to User's Guide
reference pages. Will wait a day or two for you to update the man
pages, unless you would prefer to have me convert and then make your
changes directly in sgml.

I'm doing a small reorg on the Admin Guide. Most chapters stay the
same, but things are streamlined and flow better.

                     - Thomas

-- 
Thomas Lockhart				lockhart@alumni.caltech.edu
South Pasadena, California

Thu, 20 May 1999 03:53:47 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Re: [HACKERS] Last call for docs
Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
>> Is there any equivalent in the SGML docs to the postmaster.1 and
>> postgres.1 man pages (specifically, doco for the postmaster/backend
>> command line switches)?  I couldn't find it but maybe it's there...

> No, but I want to add them by converting the man pages to User's Guide
> reference pages. Will wait a day or two for you to update the man
> pages,

Already committed my changes to the .1 files; you may fire when ready.
(Unless anyone else has updates to make there?)

			regards, tom lane

Thu, 20 May 1999 09:14:15 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: [HACKERS] Last call for docs 