Would it be a good idea to have a separate shared buffer process to
manage the cache?  Could such a process take workload off of the main
backend and improve their performance?

Just an idea?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Fri, 26 Sep 2003 22:01:33 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Separate shared_buffer management process
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Would it be a good idea to have a separate shared buffer process to
> manage the cache?  Could such a process take workload off of the main
> backend and improve their performance?

> Just an idea?

I can't recall if this has been discussed on the list, but I know I've
thought about the idea of a background "buffer writer" process that
would simply cycle through the buffer cache and write out dirty buffers
in some low-priority fashion.

The idea is this would reduce the I/O crunch at checkpoint times, as
well as reducing the odds that any foreground backend process would have
to block waiting for I/O before it could recycle a buffer slot to read
in a page it needs.  (Perhaps the background writer could be tuned to
preferentially write dirty buffers that are near the tail of the LRU
queue, and thus are likely to get recycled soon.)

In the WAL world, you cannot "write a dirty buffer" until you have
written *and synced* the WAL log as least as far as the LSN of the
buffer you want to write.  So a background buffer writer would have
to write WAL buffers as well, and in that context it could find itself
blocking foreground processes.  I'm not sure what this does to the
notion of "background I/O".  Maybe only buffers whose changes are
already synced in WAL should be eligible for background write.
It needs some thought anyway.

			regards, tom lane

Fri, 26 Sep 2003 22:44:43 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Separate shared_buffer management process 

Added to TODO:

	* Use background process to write dirty shared buffers to disk

---------------------------------------------------------------------------

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Would it be a good idea to have a separate shared buffer process to
> > manage the cache?  Could such a process take workload off of the main
> > backend and improve their performance?
> 
> > Just an idea?
> 
> I can't recall if this has been discussed on the list, but I know I've
> thought about the idea of a background "buffer writer" process that
> would simply cycle through the buffer cache and write out dirty buffers
> in some low-priority fashion.
> 
> The idea is this would reduce the I/O crunch at checkpoint times, as
> well as reducing the odds that any foreground backend process would have
> to block waiting for I/O before it could recycle a buffer slot to read
> in a page it needs.  (Perhaps the background writer could be tuned to
> preferentially write dirty buffers that are near the tail of the LRU
> queue, and thus are likely to get recycled soon.)
> 
> In the WAL world, you cannot "write a dirty buffer" until you have
> written *and synced* the WAL log as least as far as the LSN of the
> buffer you want to write.  So a background buffer writer would have
> to write WAL buffers as well, and in that context it could find itself
> blocking foreground processes.  I'm not sure what this does to the
> notion of "background I/O".  Maybe only buffers whose changes are
> already synced in WAL should be eligible for background write.
> It needs some thought anyway.
> 
> 			regards, tom lane
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Wed, 8 Oct 2003 15:33:44 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: Separate shared_buffer management process