  Date: Wednesday, July  5, 2000 @ 12:17:41
Author: wieck

Update of /home/projects/pgsql/cvsroot/pgsql/src/backend/commands
     from hub.org:/tmp/cvs-serv21402/backend/commands

Modified Files:
	command.c vacuum.c 

-----------------------------  Log Message  -----------------------------

Changed TOAST relations to have relkind RELKIND_TOASTVALUE.

Special handling of TOAST relations during VACUUM. TOAST relations
are vacuumed while the lock on the master table is still active.
The ANALYZE flag doesn't propagate to their vacuuming because the
toaster access routines allways use index access ignoring stats, so
why compute them at all.

Protection of TOAST relations against normal INSERT/UPDATE/DELETE
while offering SELECT for debugging purposes.

Jan


Wed, 5 Jul 2000 12:17:43 -0400 (EDT)
Jan Wieck <wieck>
pgsql/src/backend/commands (command.c vacuum.c)
> -----Original Message-----
> From: Jan Wieck
> Sent: Thursday, July 06, 2000 1:18 AM
> To: pgsql-committers@postgresql.org
> Subject: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c)
> 
> 
>   Date: Wednesday, July  5, 2000 @ 12:17:41
> Author: wieck
> 
> Update of /home/projects/pgsql/cvsroot/pgsql/src/backend/commands
>      from hub.org:/tmp/cvs-serv21402/backend/commands
> 
> Modified Files:
> 	command.c vacuum.c 
> 
> -----------------------------  Log Message  -----------------------------
> 
> Changed TOAST relations to have relkind RELKIND_TOASTVALUE.
> 
> Special handling of TOAST relations during VACUUM. TOAST relations
> are vacuumed while the lock on the master table is still active.
> 

It seems very dangerous to me.
When VACUUM of a master table was finished, the transaction is
in already committed state in many cases. 

Regards.
Hiroshi Inoue

Sat, 9 Dec 2000 22:23:22 +0900
"Hiroshi Inoue" <Inoue@tpf.co.jp>
RE: pgsql/src/backend/commands (command.c vacuum.c)
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
>> Special handling of TOAST relations during VACUUM. TOAST relations
>> are vacuumed while the lock on the master table is still active.

> It seems very dangerous to me.
> When VACUUM of a master table was finished, the transaction is
> in already committed state in many cases. 

I don't see the problem.  If the toast table doesn't get vacuumed,
no real harm is done other than failing to recover space.

			regards, tom lane

Sat, 09 Dec 2000 10:52:05 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: pgsql/src/backend/commands (command.c vacuum.c) 
> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
> 
> "Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> >> Special handling of TOAST relations during VACUUM. TOAST relations
> >> are vacuumed while the lock on the master table is still active.
> 
> > It seems very dangerous to me.
> > When VACUUM of a master table was finished, the transaction is
> > in already committed state in many cases. 
> 
> I don't see the problem.  If the toast table doesn't get vacuumed,
> no real harm is done other than failing to recover space.
> 

Hmm,is there any good reason to vacuum toast table in the 
transaction which was already internally committed by vacuum
of the master table ?  Is it possible under WAL ?

Regards.
Hiroshi Inoue

Sun, 10 Dec 2000 08:25:24 +0900
"Hiroshi Inoue" <Inoue@tpf.co.jp>
RE: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c) 
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> Hmm,is there any good reason to vacuum toast table in the 
> transaction which was already internally committed by vacuum
> of the master table ?  Is it possible under WAL ?

It had better be possible under WAL, because vacuuming indexes is
done in essentially the same way: we clean the indexes *after* we
commit the master's tuple movements.

Really, the TOAST table is being treated the same way we handle
indexes, and I think that's good.

			regards, tom lane

Sat, 09 Dec 2000 18:58:01 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c) 
> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
> 
> "Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> > Hmm,is there any good reason to vacuum toast table in the 
> > transaction which was already internally committed by vacuum
> > of the master table ?  Is it possible under WAL ?
> 
> It had better be possible under WAL, because vacuuming indexes is
> done in essentially the same way: we clean the indexes *after* we
> commit the master's tuple movements.
> 

There's no command other than VACUUM which continues to
access table/index after *commit*. We couldn't process
significant procedures in such an already commiitted state,
could we ? 

> Really, the TOAST table is being treated the same way we handle
> indexes, and I think that's good.
>

If I recognize correctly,TOAST table is a table not an index and
is little different from ordinary tables. VACUUM now vacuums
2 tables in a transaction for tables with TOAST columns.
          ^^^^^^^^^^^^^^^^^^
I don't think it's right and my question is simple.
What's wrong with vacuuming master and the toast table in
separate transactions ?

Regrads.
Hiroshi Inoue

Sun, 10 Dec 2000 22:48:12 +0900
"Hiroshi Inoue" <Inoue@tpf.co.jp>
RE: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c) 
"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> There's no command other than VACUUM which continues to
> access table/index after *commit*. We couldn't process
> significant procedures in such an already commiitted state,
> could we ? 

Why not?  The intermediate state *is valid*.  We just haven't
removed no-longer-referenced index and TOAST entries yet.

> What's wrong with vacuuming master and the toast table in
> separate transactions ?

You'd have to give up the lock on the master table if there were
a true commit.  I don't want to do that ... especially not when
I don't believe there is a problem to fix.

			regards, tom lane

Sun, 10 Dec 2000 12:12:50 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c) 


Tom Lane wrote:
> 
> "Hiroshi Inoue" <Inoue@tpf.co.jp> writes:
> > There's no command other than VACUUM which continues to
> > access table/index after *commit*. We couldn't process
> > significant procedures in such an already commiitted state,
> > could we ?
> 
> Why not?  The intermediate state *is valid*.  We just haven't
> removed no-longer-referenced index and TOAST entries yet.
>

Do you mean *already committed* state has no problem and  
VACUUM is always possible in the state ?
Is VACUUM such a trivial job ?

> > What's wrong with vacuuming master and the toast table in
> > separate transactions ?
> 
> You'd have to give up the lock on the master table if there were
> a true commit.  I don't want to do that ... especially not when
> I don't believe there is a problem to fix.
> 

Hmmm,is keeping the lock on master table more important than
risking to break consistency ?

Regards.
Hiroshi Inoue

Mon, 11 Dec 2000 09:00:03 +0900
Hiroshi Inoue <Inoue@tpf.co.jp>
Re: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c)
Hiroshi Inoue <Inoue@tpf.co.jp> writes:
> Tom Lane wrote:
>> Why not?  The intermediate state *is valid*.  We just haven't
>> removed no-longer-referenced index and TOAST entries yet.

> Do you mean *already committed* state has no problem and  
> VACUUM is always possible in the state ?

Yes.  Otherwise VACUUM wouldn't be crash-safe.

> Hmmm,is keeping the lock on master table more important than
> risking to break consistency ?

I see no consistency risk here.  I'd be more worried about potential
risks from dropping the lock too soon.

			regards, tom lane

Sun, 10 Dec 2000 19:08:35 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c) 
Tom Lane wrote:
> 
> Hiroshi Inoue <Inoue@tpf.co.jp> writes:
> > Tom Lane wrote:
> >> Why not?  The intermediate state *is valid*.  We just haven't
> >> removed no-longer-referenced index and TOAST entries yet.
> 
> > Do you mean *already committed* state has no problem and
> > VACUUM is always possible in the state ?
> 
> Yes.  Otherwise VACUUM wouldn't be crash-safe.
>

When VACUUM for a table starts, the transaction is not
committed yet of cource. After *commit* VACUUM has handled
heap/index tuples very carefully to be crash-safe before
7.1. Currently another vacuum could be invoked in the
already committed transaction. There has been no such
situation before 7.1. Yes,VACUUM isn't crash-safe now.
 
> > Hmmm,is keeping the lock on master table more important than
> > risking to break consistency ?
> 
> I see no consistency risk here.  I'd be more worried about potential
> risks from dropping the lock too soon.
> 

Thers's no potential risk other than deadlock.
If we have to avoid deadlock we could acquire
the lock on master table first. Is there any 
problem ?

Regards.
Hiroshi Inoue

Mon, 11 Dec 2000 12:08:25 +0900
Hiroshi Inoue <Inoue@tpf.co.jp>
Re: [COMMITTERS] pgsql/src/backend/commands (command.c vacuum.c)
Hiroshi Inoue <Inoue@tpf.co.jp> writes:
> When VACUUM for a table starts, the transaction is not
> committed yet of cource. After *commit* VACUUM has handled
> heap/index tuples very carefully to be crash-safe before
> 7.1. Currently another vacuum could be invoked in the
> already committed transaction. There has been no such
> situation before 7.1. Yes,VACUUM isn't crash-safe now.
 
Vadim, do you agree with this argument?  If so, I think it's
something we need to fix.  I don't see what Hiroshi is worried
about, myself, but if there really is an issue here...

			regards, tom lane

Mon, 11 Dec 2000 11:57:49 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Is VACUUM still crash-safe?