
> > ... most file systems can't process fsync's
> > simultaneous with other writes, so those writes block because the file
> > system grabs its own internal locks.
> 
> Oh?  That would be a serious problem, but I've never heard that asserted
> before.  Please provide some evidence.
> 
> On a filesystem that does have that kind of problem, can't you avoid it
> just by using O_DSYNC on the WAL files?

To make this competitive, the WAL writes would need to be improved to 
do more than one block (up to 256k or 512k per write) with one write call 
(if that much is to be written for this tx to be able to commit).
This should actually not be too difficult since the WAL buffer is already 
contiguous memory.

If that is done, then I bet O_DSYNC will beat any other config we currently 
have.

With this, a separate disk for WAL and large transactions you shoud be able 
to see your disks hit the max IO figures they are capable of :-)

Andreas

Fri, 4 Oct 2002 23:14:56 +0200
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at>
Re: Potential Large Performance Gain in WAL synching 
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:
> To make this competitive, the WAL writes would need to be improved to 
> do more than one block (up to 256k or 512k per write) with one write call 
> (if that much is to be written for this tx to be able to commit).
> This should actually not be too difficult since the WAL buffer is already 
> contiguous memory.

Hmmm ... if you were willing to dedicate a half meg or meg of shared
memory for WAL buffers, that's doable.  I was originally thinking of
having the (still hypothetical) background process wake up every time a
WAL page was completed and available to write.  But it could be set up
so that there is some "slop", and it only wakes up when the number of
writable pages exceeds N, for some N that's still well less than the
number of buffers.  Then it could write up to N sequential pages in a
single write().

However, this would only be a win if you had few and large transactions.
Any COMMIT will force a write of whatever we have so far, so the idea of
writing hundreds of K per WAL write can only work if it's hundreds of K
between commit records.  Is that a common scenario?  I doubt it.

If you try to set it up that way, then it's more likely that what will
happen is the background process seldom awakens at all, and each
committer effectively becomes responsible for writing all the WAL
traffic since the last commit.  Wouldn't that lose compared to someone
else having written the previous WAL pages in background?

We could certainly build the code to support this, though, and then
experiment with different values of N.  If it turns out N==1 is best
after all, I don't think we'd have wasted much code.

			regards, tom lane

Fri, 04 Oct 2002 17:43:07 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Potential Large Performance Gain in WAL synching 