> > > > I think you have identified a problem that needs
> > > > a more general solution: we need to be robust in the case that
> > > > an index entry is on disk that points to a tuple that never made
> > > > it to disk.
> > 
> > And this general solution is WAL.
> >
> Yes exactly.
> But I've thought it's mainly for aborts in the middle of btree page
> splitting or for system crash in which we couldn't expect synchronous
> flushing of dirty buffers.

Central idea of WAL - write (and flush) to log all changes made in data
buffers _before_ data files will be changed. Buffer mgmr will be
responsible for this. Changes made in table buffers will be logged before
changes made in index ones, redo will insert un-inserted table rows and
index rows will not point to unexistent tuples in table. Undo will erase
all uncommitted changes (but will not shrink tables/indices).

Vadim
 


Tue, 9 May 2000 17:06:30 -0700 
"Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
RE: Shouldn't flush dirty buffers at shutdown ? 
> -----Original Message-----
> From: Mikheev, Vadim [mailto:vmikheev@SECTORBASE.COM]
> 
> > > > > I think you have identified a problem that needs
> > > > > a more general solution: we need to be robust in the case that
> > > > > an index entry is on disk that points to a tuple that never made
> > > > > it to disk.
> > > 
> > > And this general solution is WAL.
> > >
> > Yes exactly.
> > But I've thought it's mainly for aborts in the middle of btree page
> > splitting or for system crash in which we couldn't expect synchronous
> > flushing of dirty buffers.
> 
> Central idea of WAL - write (and flush) to log all changes made in data
> buffers _before_ data files will be changed. Buffer mgmr will be
> responsible for this. Changes made in table buffers will be logged before
> changes made in index ones, redo will insert un-inserted table rows and
> index rows will not point to unexistent tuples in table. Undo will erase
> all uncommitted changes (but will not shrink tables/indices).
>

Yes WAL would naturally solve a current flaw of index updation as a part
of its effect.
What I've never understood until recently is that even normal aborts(not
in the middle of b-tree splitting) and normal shutdown could cause an
inconsistency between heap and indices. 

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

Wed, 10 May 2000 12:27:56 +0900
"Hiroshi Inoue" <Inoue@tpf.co.jp>
RE: Shouldn't flush dirty buffers at shutdown ? 
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> What I've never understood until recently is that even normal aborts(not
> in the middle of b-tree splitting) and normal shutdown could cause an
> inconsistency between heap and indices. 

Yes.  Since WAL will provide the real solution in 7.1, I think we need
only look for a simple stopgap answer for 7.0.x.  Perhaps we could just
tweak bufmgr.c so that dirty buffers are flushed out on both transaction
commit and abort.  That doesn't solve the consistency-after-crash issue,
but at least you can do an orderly shutdown of a postmaster without
fear.  Is it worth trying to do more now, rather than working on WAL?

			regards, tom lane

Tue, 09 May 2000 23:47:39 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Shouldn't flush dirty buffers at shutdown ? 
> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
>
> "Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> > What I've never understood until recently is that even normal aborts(not
> > in the middle of b-tree splitting) and normal shutdown could cause an
> > inconsistency between heap and indices.
>
> Yes.  Since WAL will provide the real solution in 7.1, I think we need
> only look for a simple stopgap answer for 7.0.x.  Perhaps we could just
> tweak bufmgr.c so that dirty buffers are flushed out on both transaction
> commit and abort.  That doesn't solve the consistency-after-crash issue,
> but at least you can do an orderly shutdown of a postmaster without
> fear.  Is it worth trying to do more now, rather than working on WAL?
>

Hmm,performance vs. consistency.
I vote for consistency this time.
However other people may prefer performance because the
consistency isn't complete in any case and 7.1 would provide
a real solution.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp


Wed, 10 May 2000 14:39:06 +0900
"Hiroshi Inoue" <Inoue@tpf.co.jp>
RE: Shouldn't flush dirty buffers at shutdown ? 
> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
>
> "Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> > What I've never understood until recently is that even normal aborts(not
> > in the middle of b-tree splitting) and normal shutdown could cause an
> > inconsistency between heap and indices.
>
> Yes.  Since WAL will provide the real solution in 7.1, I think we need
> only look for a simple stopgap answer for 7.0.x.  Perhaps we could just
> tweak bufmgr.c so that dirty buffers are flushed out on both transaction
> commit and abort.  That doesn't solve the consistency-after-crash issue,
> but at least you can do an orderly shutdown of a postmaster without
> fear.  Is it worth trying to do more now, rather than working on WAL?
>

I have another anxiety now.
As far as I see,PostgreSQL doesn't call LockBuffer() before
calling smgrwrite(). This seems to mean that smgrwrite()
could write buffers to disk which are being changed by
another backend. If the(another) backend was aborted by
some reason the buffer page would remain half-changed.

Is it well known ?  Please correct me if I'm wrong.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp


Fri, 26 May 2000 12:17:18 +0900
"Hiroshi Inoue" <Inoue@tpf.co.jp>
smgrwrite() without LockBuffer(was RE: Shouldn't flush dirty buffers
	at shutdown ?)
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> As far as I see,PostgreSQL doesn't call LockBuffer() before
> calling smgrwrite(). This seems to mean that smgrwrite()
> could write buffers to disk which are being changed by
> another backend. If the(another) backend was aborted by
> some reason the buffer page would remain half-changed.

Hmm ... looks fishy to me too.  Seems like we ought to hold
BUFFER_LOCK_SHARE on the buffer while dumping it out.  It
wouldn't matter under normal circumstances, but as you say
there could be trouble if the other backend crashed before
it could mark the buffer dirty again, or if we had a system
crash before the dirtied page got written again.

Vadim, what do you think?

			regards, tom lane

Thu, 25 May 2000 23:40:13 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: smgrwrite() without LockBuffer(was RE: Shouldn't flush dirty
	buffers at shutdown ?)