I've found a number of infelicities in the hash index code that can't be
fixed without an on-disk format change.  The biggest one is that the 
hashm_ntuples field in hash meta pages is only uint32, meaning that
hash index space management will become confused if the number of
entries exceeds 4G.  I'd like to change it to a "double", and clean up
a couple other uglinesses at the same time.

Ordinarily I'd just force an initdb for such a change, but at this late
stage of the 7.4 cycle it seems better to avoid requiring initdb,
especially since many beta testers wouldn't be using hash indexes anyway
and shouldn't need to reload.  What I intend to do instead is increment
the version number that already exists in the hash metapage, and add
code to spit out a "please reindex this index" error if the version
number isn't right.  A REINDEX command would be sufficient to
reconstruct the index in the new format.

Any objections?

			regards, tom lane

Tue, 02 Sep 2003 11:03:24 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Planning to force reindex of hash indexes
Tom Lane wrote:
> I've found a number of infelicities in the hash index code that can't be
> fixed without an on-disk format change.  The biggest one is that the 
> hashm_ntuples field in hash meta pages is only uint32, meaning that
> hash index space management will become confused if the number of
> entries exceeds 4G.  I'd like to change it to a "double", and clean up
> a couple other uglinesses at the same time.
> 
> Ordinarily I'd just force an initdb for such a change, but at this late
> stage of the 7.4 cycle it seems better to avoid requiring initdb,
> especially since many beta testers wouldn't be using hash indexes anyway
> and shouldn't need to reload.  What I intend to do instead is increment
> the version number that already exists in the hash metapage, and add
> code to spit out a "please reindex this index" error if the version
> number isn't right.  A REINDEX command would be sufficient to
> reconstruct the index in the new format.
> 
> Any objections?

Good plan.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Wed, 3 Sep 2003 17:01:58 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Planning to force reindex of hash indexes
"Tom Lane" <tgl@sss.pgh.pa.us> wrote:
> I've found a number of infelicities in the hash index code that can't be
> fixed without an on-disk format change.  The biggest one is that the 
> hashm_ntuples field in hash meta pages is only uint32, meaning that
> hash index space management will become confused if the number of
> entries exceeds 4G.  I'd like to change it to a "double", and clean up
> a couple other uglinesses at the same time.

How can we avoid this kind of mess for the future ?

Regards
Gaetano Mendola


Fri, 5 Sep 2003 17:20:49 +0200
"Mendola Gaetano" <mendola@bigfoot.com>
Re: Planning to force reindex of hash indexes
"Mendola Gaetano" <mendola@bigfoot.com> writes:
> "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
>> I've found a number of infelicities in the hash index code that can't be
>> fixed without an on-disk format change.

> How can we avoid this kind of mess for the future ?

Build a time machine, go back fifteen years, wave a magic wand to
increase the IQ levels of the Berkeley grad students?  Sometimes
we just have to change bad decisions, that's all.

			regards, tom lane

Fri, 05 Sep 2003 11:40:00 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Planning to force reindex of hash indexes 
"Tom Lane" <tgl@sss.pgh.pa.us> wrote:
> "Mendola Gaetano" <mendola@bigfoot.com> writes:
> > "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
> >> I've found a number of infelicities in the hash index code that can't
be
> >> fixed without an on-disk format change.
>
> > How can we avoid this kind of mess for the future ?
>
> Build a time machine, go back fifteen years, wave a magic wand to
> increase the IQ levels of the Berkeley grad students?

:-)

> Sometimes we just have to change bad decisions, that's all.

I don't know how much old is the code incriminated but I mean
there is no way to improve the code approved ?

Regards
Gaetano Mendola




Fri, 5 Sep 2003 18:29:12 +0200
"Mendola Gaetano" <mendola@bigfoot.com>
Re: Planning to force reindex of hash indexes
Tom Lane <tgl@sss.pgh.pa.us> writes:

> "Mendola Gaetano" <mendola@bigfoot.com> writes:
>> "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
>>> I've found a number of infelicities in the hash index code that
>>> can't be fixed without an on-disk format change.
>
>> How can we avoid this kind of mess for the future ?
>
> Build a time machine, go back fifteen years, wave a magic wand to
> increase the IQ levels of the Berkeley grad students?  Sometimes we
> just have to change bad decisions, that's all.
>
> 			regards, tom lane

Actually, I think that your time would be better spent on the time
machine.  After all, with working time travel you wouldn't have to
worry about optimizing PostgreSQL.  Queries could take as long as they
needed to take and then you could send the response back in time to
right before the query was issued.

It's too bad you don't have the time machine finished already.  You
could use the time machine to go back in time and work on the time
machine :).  You'd be finished in no time.

Jason

Fri, 05 Sep 2003 10:59:38 -0600
jearl@bullysports.com
Re: Planning to force reindex of hash indexes
Mendola Gaetano wrote:
> "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
> > "Mendola Gaetano" <mendola@bigfoot.com> writes:
> > > "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
> > >> I've found a number of infelicities in the hash index code that can't
> be
> > >> fixed without an on-disk format change.
> >
> > > How can we avoid this kind of mess for the future ?
> >
> > Build a time machine, go back fifteen years, wave a magic wand to
> > increase the IQ levels of the Berkeley grad students?
> 
> :-)
> 
> > Sometimes we just have to change bad decisions, that's all.
> 
> I don't know how much old is the code incriminated but I mean
> there is no way to improve the code approved ?

I am not sure what you are asking, but the bug existed long before we
got it from Berkeley, and even then, it might have slipped by us anyway
without our noticing anyway  --- we are never going to be perfect.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fri, 5 Sep 2003 16:40:07 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Planning to force reindex of hash indexes