Could we remove lines 552-560 of pgbench.c?  The behavior that guarded
against is long gone, and forcing a checkpoint every few thousand tuples
seems to be putting a huge crimp in the speed of pgbench -i ...

			regards, tom lane

Sun, 06 Jan 2002 00:22:33 -0500
Tom Lane <tgl@sss.pgh.pa.us>
pgbench -i spends all its time doing CHECKPOINT
Tom Lane wrote:
> Could we remove lines 552-560 of pgbench.c?  The behavior that guarded
> against is long gone, and forcing a checkpoint every few thousand tuples
> seems to be putting a huge crimp in the speed of pgbench -i ...

I don't see any need for it.  It isn't even needed for 7.1.3.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 6 Jan 2002 01:38:52 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pgbench -i spends all its time doing CHECKPOINT
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tom Lane wrote:
>> Could we remove lines 552-560 of pgbench.c?  The behavior that guarded
>> against is long gone, and forcing a checkpoint every few thousand tuples
>> seems to be putting a huge crimp in the speed of pgbench -i ...

> I don't see any need for it.

Au contraire.  I'm nearly done with pgbench -i -s 500 after about an
hour.  The unmodified code ran for several hours and was less than
half done.

			regards, tom lane

Sun, 06 Jan 2002 01:41:17 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: pgbench -i spends all its time doing CHECKPOINT 
Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Tom Lane wrote:
> >> Could we remove lines 552-560 of pgbench.c?  The behavior that guarded
> >> against is long gone, and forcing a checkpoint every few thousand tuples
> >> seems to be putting a huge crimp in the speed of pgbench -i ...
> 
> > I don't see any need for it.
> 
> Au contraire.  I'm nearly done with pgbench -i -s 500 after about an
> hour.  The unmodified code ran for several hours and was less than
> half done.

I meant I don't see any need for that checkpoint code.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 6 Jan 2002 01:44:50 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pgbench -i spends all its time doing CHECKPOINT
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I meant I don't see any need for that checkpoint code.

Oh, my mistake; I thought you meant you didn't want to make a change
now.

			regards, tom lane

Sun, 06 Jan 2002 01:50:33 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: pgbench -i spends all its time doing CHECKPOINT 
> Could we remove lines 552-560 of pgbench.c?  The behavior that guarded
> against is long gone, and forcing a checkpoint every few thousand tuples
> seems to be putting a huge crimp in the speed of pgbench -i ...

Yup. Maybe we could ifdef'ed out until we implement true UNDO...
--
Tatsuo Ishii

Sun, 06 Jan 2002 17:10:36 +0900
Tatsuo Ishii <t-ishii@sra.co.jp>
Re: pgbench -i spends all its time doing CHECKPOINT
Tatsuo Ishii wrote:
> > Could we remove lines 552-560 of pgbench.c?  The behavior that guarded
> > against is long gone, and forcing a checkpoint every few thousand tuples
> > seems to be putting a huge crimp in the speed of pgbench -i ...
> 
> Yup. Maybe we could ifdef'ed out until we implement true UNDO...

I think we should just remove it.  The idea that we are going to do UNDO
which allows unlimited log file growth for long transactions seems like
a loser to me.

Actually, that brings up a question I had. In 7.1.0, we didn't recycle
WAL segements that were used by open transactions during CHECKPOINT,
while in 7.1.3 and later, we do recycle them after CHECKPOINT.  My
question is if we do a big transaction that needs 10 log segments, do we
force an early CHECKPOINT to clear out the WAL segments or do we just
wait for the proper interval?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 6 Jan 2002 13:01:41 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pgbench -i spends all its time doing CHECKPOINT
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> question is if we do a big transaction that needs 10 log segments, do we
> force an early CHECKPOINT to clear out the WAL segments or do we just
> wait for the proper interval?

A checkpoint is forced after every CHECKPOINT_SEGMENTS log segments,
regardless of longevity of transactions.  See
http://developer.postgresql.org/docs/postgres/wal-configuration.html

Since segments before the checkpoint-before-last are deleted or recycled
after each checkpoint, the maximum number of back segments would
normally be 2 * CHECKPOINT_SEGMENTS.  We also pre-create WAL_FILES
future log segments.  Counting the current segment gives a total of
WAL_FILES + 2 * CHECKPOINT_SEGMENTS + 1 log segments.

AFAICS, the only way to force the current code into creating more than
WAL_FILES + 2 * CHECKPOINT_SEGMENTS + 1 log segments is to be generating
WAL entries at such a high rate that more than WAL_FILES log segments
are filled before a triggered checkpoint can be completed.

			regards, tom lane

Sun, 06 Jan 2002 13:37:34 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: pgbench -i spends all its time doing CHECKPOINT 
Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > question is if we do a big transaction that needs 10 log segments, do we
> > force an early CHECKPOINT to clear out the WAL segments or do we just
> > wait for the proper interval?
> 
> A checkpoint is forced after every CHECKPOINT_SEGMENTS log segments,
> regardless of longevity of transactions.  See
> http://developer.postgresql.org/docs/postgres/wal-configuration.html
> 
> Since segments before the checkpoint-before-last are deleted or recycled
> after each checkpoint, the maximum number of back segments would
> normally be 2 * CHECKPOINT_SEGMENTS.  We also pre-create WAL_FILES
> future log segments.  Counting the current segment gives a total of
> WAL_FILES + 2 * CHECKPOINT_SEGMENTS + 1 log segments.
> 
> AFAICS, the only way to force the current code into creating more than
> WAL_FILES + 2 * CHECKPOINT_SEGMENTS + 1 log segments is to be generating
> WAL entries at such a high rate that more than WAL_FILES log segments
> are filled before a triggered checkpoint can be completed.
> 

Very interesting.  Thanks.  Is there a reason someone would manually run
the CHECKPOINT command?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Sun, 6 Jan 2002 14:47:04 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pgbench -i spends all its time doing CHECKPOINT