> >> This is OK for table files, unless someone's broken the 
> >> code that will auto-initialize a zero page when it comes across one.
> 
> > Hmmm, I don't see anything like auto-initialization in code -:(
> > Where did you put these changes?
> 
> I didn't put 'em in, it looked like your work to me: see vacuum.c,
> lines 618-622 in current sources.

Oh, this code was there from 6.0 days.

> Awhile back I did fix PageGetFreeSpace and some related macros to
> deliver sane results when looking at an all-zero page header, so that
> scans and inserts would ignore the page until vacuum fixes it.

I see now - PageGetMaxOffsetNumber... Ok.
 
> Perhaps WAL redo needs to be prepared to do PageInit as well?

It calls PageIsNew and uses flag in record to know when a page could
be uninitialized.
 
> Actually, I'd expect the CRC check to catch an all-zeroes page (if
> it fails to complain, then you misimplemented the CRC), so that would
> be the place to deal with it now.

I've used standard CRC32 implementation you pointed me to -:)
But CRC is used in WAL records only.

Vadim

Thu, 11 Jan 2001 17:19:29 -0800
"Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
RE: Re: Loading optimization 
"Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
>> Actually, I'd expect the CRC check to catch an all-zeroes page (if
>> it fails to complain, then you misimplemented the CRC), so that would
>> be the place to deal with it now.

> I've used standard CRC32 implementation you pointed me to -:)
> But CRC is used in WAL records only.

Oh.  I thought we'd agreed that a CRC on each stored disk block would
be a good idea as well.  I take it you didn't do that.

Do we want to consider doing this (and forcing another initdb)?
Or shall we say "too late for 7.1"?

			regards, tom lane

Thu, 11 Jan 2001 21:55:58 -0500
Tom Lane <tgl@sss.pgh.pa.us>
CRCs (was Re: [GENERAL] Re: Loading optimization)
> "Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> >> Actually, I'd expect the CRC check to catch an all-zeroes page (if
> >> it fails to complain, then you misimplemented the CRC), so that would
> >> be the place to deal with it now.
> 
> > I've used standard CRC32 implementation you pointed me to -:)
> > But CRC is used in WAL records only.
> 
> Oh.  I thought we'd agreed that a CRC on each stored disk block would
> be a good idea as well.  I take it you didn't do that.


No, I thought we agreed disk block CRC was way overkill.  If the CRC on
the WAL log checks for errors that are not checked anywhere else, then
fine, but I thought disk CRC would just duplicate the I/O subsystem/disk
checks.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Fri, 12 Jan 2001 00:39:23 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: CRCs (was Re: [GENERAL] Re: Loading optimization)
Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> Oh.  I thought we'd agreed that a CRC on each stored disk block would
>> be a good idea as well.  I take it you didn't do that.

> No, I thought we agreed disk block CRC was way overkill.  If the CRC on
> the WAL log checks for errors that are not checked anywhere else, then
> fine, but I thought disk CRC would just duplicate the I/O subsystem/disk
> checks.

A disk-block CRC would detect partially written blocks (ie, power drops
after disk has written M of the N sectors in a block).  The disk's own
checks will NOT consider this condition a failure.  I'm not convinced
that WAL will reliably detect it either (Vadim?).  Certainly WAL will
not help for corruption caused by external agents, away from any updates
that are actually being performed/logged.

			regards, tom lane

Fri, 12 Jan 2001 01:16:20 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: CRCs (was Re: [GENERAL] Re: Loading optimization) 
At 21:55 11/01/01 -0500, Tom Lane wrote:
>
>Oh.  I thought we'd agreed that a CRC on each stored disk block would
>be a good idea as well.  I take it you didn't do that.
>
>Do we want to consider doing this (and forcing another initdb)?
>Or shall we say "too late for 7.1"?
>

I thought it was coming too. I'd like to see it - if it's not too hard in
this release.


----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/

Fri, 12 Jan 2001 17:20:22 +1100
Philip Warner <pjw@rhyme.com.au>
Re: CRCs (was Re: [GENERAL] Re: Loading optimization)
> > But CRC is used in WAL records only.
> 
> Oh.  I thought we'd agreed that a CRC on each stored disk block would
> be a good idea as well.  I take it you didn't do that.
> 
> Do we want to consider doing this (and forcing another initdb)?
> Or shall we say "too late for 7.1"?

I personally was never agreed to this. Reasons?

Vadim



Thu, 11 Jan 2001 23:07:16 -0800
"Vadim Mikheev" <vmikheev@sectorbase.com>
Re: CRCs (was Re: [GENERAL] Re: Loading optimization)
> > No, I thought we agreed disk block CRC was way overkill.  If the CRC on
> > the WAL log checks for errors that are not checked anywhere else, then
> > fine, but I thought disk CRC would just duplicate the I/O subsystem/disk
> > checks.
> 
> A disk-block CRC would detect partially written blocks (ie, power drops
> after disk has written M of the N sectors in a block).  The disk's own
> checks will NOT consider this condition a failure.  I'm not convinced
> that WAL will reliably detect it either (Vadim?).  Certainly WAL will

Idea proposed by Andreas about "physical log" is implemented!
Now WAL saves whole data blocks on first after checkpoint
modification. This way on recovery modified data blocks will be
first restored *as a whole*. Isn't it much better than just
detection of partially writes?

Only one type of modification isn't covered at the moment -
updated t_infomask of heap tuples.

> not help for corruption caused by external agents, away from any updates
> that are actually being performed/logged.

What do you mean by "external agents"?

Vadim



Thu, 11 Jan 2001 23:32:12 -0800
"Vadim Mikheev" <vmikheev@sectorbase.com>
Re: CRCs (was Re: [GENERAL] Re: Loading optimization) 