I have just sent to the pgsql-patches list a rather large set of
proposed diffs for the WAL code.  These changes:

* Store two past checkpoint locations, not just one, in pg_control.
  On startup, we fall back to the older checkpoint if the newer one
  is unreadable.  Also, a physical copy of the newest checkpoint record
  is kept in pg_control for possible use in disaster recovery (ie,
  complete loss of pg_xlog).  Also add a version number for pg_control
  itself.  Remove archdir from pg_control; it ought to be a GUC
  parameter, not a special case (not that it's implemented yet anyway).

* Suppress successive checkpoint records when nothing has been entered
  in the WAL log since the last one.  This is not so much to avoid I/O
  as to make it actually useful to keep track of the last two
  checkpoints.  If the things are right next to each other then there's
  not a lot of redundancy gained...

* Change CRC scheme to a true 64-bit CRC, not a pair of 32-bit CRCs
  on alternate bytes.

* Fix XLOG record length handling so that it will work at BLCKSZ = 32k.

* Change XID allocation to work more like OID allocation, so that we
  can flush XID alloc info to the log before there is any chance an XID
  will appear in heap files.

* Fix a number of minor bugs, such as off-by-one logic for XLOG file
  wraparound at the 4 gig mark.

* Add documentation and clean up some coding infelicities; move file
  format declarations out to include files where planned contrib
  utilities can get at them.


Before committing this stuff, I intend to prepare a contrib utility that
can be used to reset pg_control and pg_xlog.  This is mainly for
disaster recovery purposes, but as a side benefit it will allow people
to update 7.1beta installations to this new code without doing initdb.
I need to update contrib/pg_controldata, too.

			regards, tom lane

Mon, 05 Mar 2001 23:39:17 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Proposed WAL changes
> -----Original Message-----
> From: Tom Lane
> 
> I have just sent to the pgsql-patches list a rather large set of
> proposed diffs for the WAL code.  These changes:
> 
> * Store two past checkpoint locations, not just one, in pg_control.
>   On startup, we fall back to the older checkpoint if the newer one
>   is unreadable.  Also, a physical copy of the newest checkpoint record
>   is kept in pg_control for possible use in disaster recovery (ie,
>   complete loss of pg_xlog).  Also add a version number for pg_control
>   itself.  Remove archdir from pg_control; it ought to be a GUC
>   parameter, not a special case (not that it's implemented yet anyway).
> 

Is archdir really a GUC parameter ?

Regards,
Hiroshi Inoue 

Tue, 6 Mar 2001 23:55:35 +0900
"Hiroshi Inoue" <Inoue@tpf.co.jp>
RE: Proposed WAL changes
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
>> Remove archdir from pg_control; it ought to be a GUC
>> parameter, not a special case (not that it's implemented yet anyway).

> Is archdir really a GUC parameter ?

Why shouldn't it be?  I see nothing wrong with changing it on-the-fly.

			regards, tom lane

Tue, 06 Mar 2001 10:33:38 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: Proposed WAL changes 
Please forgive me if I'm misunderstanding something of these rather
complex issues.  But I think this is an important question from the
perspective of a sytem administrator responsible for the safety and
uncorruptedness of his users' data.

If I understand correctly, it is possible, through power failure,
Postgres or OS crash, or disk failure, to end up with a situation
where Postgres cannot put the database in a consistent state.  Rather
than failing to start at all, something will be put in place that
allows PG to partially recover and start up, so that you can get to
your data.  I think that leaves DBAs wondering two things:

First, how will I know that my database is corrupted?  I may not be
present to witness the power failure, unattended reboot, and automatic
restart/quasi-recovery.  If the DB has become inconsistent, it is
critical that users do not continue to use it.  I'm all for having
Postgres throw up its hands and refuse to start until I put it in
disaster-dump mode.

Secondly, since disaster-dump seems to be a good term for it, is there
some way to know the extent of the damage?  I.e. in the typical case
of power failure, is the inconsistent part just the "recent" data,
and would it be possible to find out which records are part of that
damaged set?
-- 
Christopher Masto         Senior Network Monkey      NetMonger Communications
chris@netmonger.net        info@netmonger.net        http://www.netmonger.net

Free yourself, free your machine, free the daemon -- http://www.freebsd.org/

Thu, 8 Mar 2001 17:47:48 -0500
Christopher Masto <chris@netmonger.net>
Re: Proposed WAL changes