> >> What I was thinking about in that last paragraph was manual 
> >> analysis and recovery. I don't think it's a good idea for automatic
> >> system startup to skip over gaps in the log.
> 
> > But if we'll not try to read after gap then after restart 
> > system will not notice gap and valid records after it and
> > just rewrite log space with new records. Not much chance for
> > manual analysis - ppl will not report any problems.
> 
> That'll be true in any case, unless we refuse to start up at all upon
> detecting xlog corruption (which doesn't seem like the way to fly).
> Not sure what we can do about that.

What I would refuse in the event of log corruption is continuing
normal database operations. It's ok to dump such database for manual
recovery, but continuing to use it is VERY BAD THING. The fact that
users will use inconsistent DB may become big headache for us - just
imagine compains about index scans returning incorrect results
(index tuples pointing to free heap space was left and then that space
was used for tuple with different keys).

Failing to restart was bad but silent restart in the event of log
corruption is bad too. In first case we had at least chance to
discover original problem.

Vadim

Tue, 13 Mar 2001 10:39:47 -0800
"Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
RE: xlog loose ends, continued 
"Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
>> That'll be true in any case, unless we refuse to start up at all upon
>> detecting xlog corruption (which doesn't seem like the way to fly).
>> Not sure what we can do about that.

> What I would refuse in the event of log corruption is continuing
> normal database operations.

Hmm.  We could do that if we had some notion of a read-only operating
mode, perhaps.  But we don't have one now and I don't want to add it
for 7.1.  Can we agree to look at this more for 7.2?

If we did have that, it would make sense to scan the rest of the log
(after the last valid XLOG record) to see if we find any more records.
If we do then --- whether they're valid or not --- we have a corrupted
DB and we should go into the read-only state.  But for the moment I
think it's best not to make such a scan.

			regards, tom lane

Tue, 13 Mar 2001 13:53:42 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: xlog loose ends, continued 
> >> That'll be true in any case, unless we refuse to start up at all upon
> >> detecting xlog corruption (which doesn't seem like the way to fly).
> >> Not sure what we can do about that.
> > What I would refuse in the event of log corruption is continuing
> > normal database operations.
> Hmm.  We could do that if we had some notion of a read-only operating
> mode, perhaps.  But we don't have one now and I don't want to add it
> for 7.1.  Can we agree to look at this more for 7.2?

I'd like to have a readonly mode driven by integrity requirements for
corruption recovery for database tables, for replication, and (in the
future) for distributed databases, so perhaps we can do a trial
implementation fairly soon. Not sure how it would impact the backend(s),
but istm that we might be able to do a first implementation for 7.1.x.
I'll bring it up again when appropriate...

                   - Thomas

Wed, 14 Mar 2001 00:39:08 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Re: xlog loose ends, continued
"Mikheev, Vadim" wrote:
> 
> > >> What I was thinking about in that last paragraph was manual
> > >> analysis and recovery. I don't think it's a good idea for automatic
> > >> system startup to skip over gaps in the log.
> >
> > > But if we'll not try to read after gap then after restart
> > > system will not notice gap and valid records after it and
> > > just rewrite log space with new records. Not much chance for
> > > manual analysis - ppl will not report any problems.
> >
> > That'll be true in any case, unless we refuse to start up at all upon
> > detecting xlog corruption (which doesn't seem like the way to fly).
> > Not sure what we can do about that.
> 
> What I would refuse in the event of log corruption is continuing
> normal database operations. 

Log corruption is never an unique cause of a recovery failure.
If there's a bug in redo stuff the result would also be a recovery
failure. Currently the redo stuff has to accomplish redo operations
completely. No matter how trivial the bug may be, it's always serious
unfortunately.

> It's ok to dump such database for manual
> recovery, but continuing to use it is VERY BAD THING. The fact that
> users will use inconsistent DB may become big headache for us - just

> imagine compains about index scans returning incorrect results
> (index tuples pointing to free heap space was left and then that space
> was used for tuple with different keys).
> 

Hmm this seems nothing worse than 7.0.
I would complain if postmaster couldn't restart due to this reason.
IIRC few ppl mind the (even system) index corruption.
 
> Failing to restart was bad but silent restart in the event of log
> corruption is bad too.

Agreed.

regards,
Hiroshi Inoue

Wed, 14 Mar 2001 11:42:11 +0900
Hiroshi Inoue <Inoue@tpf.co.jp>
Re: RE: xlog loose ends, continued
Tom Lane wrote:
> 
> "Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> >> That'll be true in any case, unless we refuse to start up at all upon
> >> detecting xlog corruption (which doesn't seem like the way to fly).
> >> Not sure what we can do about that.
> 
> > What I would refuse in the event of log corruption is continuing
> > normal database operations.
> 
> Hmm.  We could do that if we had some notion of a read-only operating
> mode, perhaps.  But we don't have one now and I don't want to add it
> for 7.1.  Can we agree to look at this more for 7.2?

I'd love to see PostgreSQL have a read-only mode of some kind that let
enquiry function against a possibly otherwise corrupted database,
without the stress of worrying that you might be making things worse.

I know other DB servers that have this sort of thing, and it has been a
life-saver for me on occasion to allow critical information to be
extracted before you nuke it all and start over.

Cheers,
				Andrew.
-- 
_____________________________________________________________________
           Andrew McMillan, e-mail: Andrew@catalyst.net.nz
Catalyst IT Ltd, PO Box 10-225, Level 22, 105 The Terrace, Wellington
Me: +64 (21) 635 694, Fax: +64 (4) 499 5596, Office: +64 (4) 499 2267

Wed, 14 Mar 2001 18:13:35 +1300
Andrew McMillan <andrew@catalyst.net.nz>
Re: Re: xlog loose ends, continued