The attached patch applies to CVS tip as of 02/05/2004 and implements 
the cost based vacuum delay feature.

A detailed description with charts of different configuration settings 
can be found here:

     http://developer.postgresql.org/~wieck/vacuum_cost/

There is a problem left that seems to be related to Toms observations in 
the shutdown behaviour of the postmaster. My current guess is that the 
napping done via select(2) somehow prevents responding to the query 
abort signal.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #
Thu, 05 Feb 2004 09:05:53 -0500
Jan Wieck <JanWieck@Yahoo.com>
Vacuum Delay feature
Attached is a corrected version that solves the query cancel problem by 
not napping any more and going full speed as soon as any signal is 
pending. If nobody objects, I'm going to commit this tomorrow.


Jan


Jan Wieck wrote:

> The attached patch applies to CVS tip as of 02/05/2004 and implements 
> the cost based vacuum delay feature.
> 
> A detailed description with charts of different configuration settings 
> can be found here:
> 
>      http://developer.postgresql.org/~wieck/vacuum_cost/
> 
> There is a problem left that seems to be related to Toms observations in 
> the shutdown behaviour of the postmaster. My current guess is that the 
> napping done via select(2) somehow prevents responding to the query 
> abort signal.
> 
> 
> Jan
> 


-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #
Thu, 05 Feb 2004 14:18:18 -0500
Jan Wieck <JanWieck@Yahoo.com>
Re: Vacuum Delay feature
Bruce Momjian wrote:

> Jan Wieck wrote:
>> Attached is a corrected version that solves the query cancel problem by 
>> not napping any more and going full speed as soon as any signal is 
>> pending. If nobody objects, I'm going to commit this tomorrow.
> 
> Jan, three questions.  First, is this useful now that we have the new
> cache replacement code, second, do we need this many parameters (can't
> any of them be autotuned), and third, what about documentation?
> 

You mean if stopping to nap is useful when a signal is pending or if 
napping during vacuum itself is useful at all?

I am willing to make it all self tuning and automagic. Just tell me how.

Documentation is missing so far. Will work on that.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #


Thu, 12 Feb 2004 15:22:28 -0500
Jan Wieck <JanWieck@Yahoo.com>
Re: [HACKERS] Vacuum Delay feature
Jan Wieck wrote:
> Attached is a corrected version that solves the query cancel problem by 
> not napping any more and going full speed as soon as any signal is 
> pending. If nobody objects, I'm going to commit this tomorrow.

Jan, three questions.  First, is this useful now that we have the new
cache replacement code, second, do we need this many parameters (can't
any of them be autotuned), and third, what about documentation?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Feb 2004 15:38:35 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] Vacuum Delay feature
Jan Wieck wrote:
> Bruce Momjian wrote:
> 
> > Jan Wieck wrote:
> >> Attached is a corrected version that solves the query cancel problem by 
> >> not napping any more and going full speed as soon as any signal is 
> >> pending. If nobody objects, I'm going to commit this tomorrow.
> > 
> > Jan, three questions.  First, is this useful now that we have the new
> > cache replacement code, second, do we need this many parameters (can't
> > any of them be autotuned), and third, what about documentation?
> > 
> 
> You mean if stopping to nap is useful when a signal is pending or if 
> napping during vacuum itself is useful at all?
> 
> I am willing to make it all self tuning and automagic. Just tell me how.

I was hoping you would have some ideas.  :-)

I guess my question is that now that we have the new cache replacement
policy, is the vacuum delay worth while.  I looked at
http://developer.postgresql.org/~wieck/vacuum_cost/ and does seem
useful.

> Documentation is missing so far. Will work on that.

Cool.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thu, 12 Feb 2004 21:03:53 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] Vacuum Delay feature
Centuries ago, Nostradamus foresaw when pgman@candle.pha.pa.us (Bruce Momjian) would write:
> I guess my question is that now that we have the new cache
> replacement policy, is the vacuum delay worth while.  I looked at
> http://developer.postgresql.org/~wieck/vacuum_cost/ and does seem
> useful.

They satisfy quite separate use cases, so both are surely useful.

- The new cache replacement policy allows us to make sure that cache
  isn't getting blown on worthless things.

- Vacuum delay allows us to make sure that we aren't spending all our
  I/O on vacuuming.

There is overlap between their uses, as both should help diminish the
use of I/O to fill buffers with data that was discarded, but they
surely have separate uses.
-- 
"cbbrowne","@","acm.org"
http://www.ntlug.org/~cbbrowne/lisp.html
You  know  that  little  indestructible  black box  that  is  used  on
planes---why  can't  they  make  the  whole  plane  out  of  the  same
substance?

Thu, 12 Feb 2004 23:20:53 -0500
Christopher Browne <cbbrowne@acm.org>
Re: Vacuum Delay feature