This might seem a bit silly, but is there any chance we could add a 
comment at the end of pg_dump text output that says '-- End of dump'?

Would make it useful for checking that you actually have a complete dump...

Chris


Tue, 30 Mar 2004 11:37:45 +0800
Christopher Kings-Lynne <chriskl@familyhealth.com.au>
pg_dump end comment
Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
> This might seem a bit silly, but is there any chance we could add a 
> comment at the end of pg_dump text output that says '-- End of dump'?

Sure --- while you're at it, put a "beginning of dump" at the start.

Is it worth adding the database name and/or other command-line
parameters given to pg_dump?

One thing to think about is the difference between a bare pg_dump and a
pg_dump/pg_restore sequence.  Should these always generate identical
text output?  (They do as of CVS tip, I believe, though this was not
always true before.)

Possibly this is all gilding the lily though...

			regards, tom lane

Tue, 30 Mar 2004 00:29:00 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: pg_dump end comment 
Tom Lane wrote:
> Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
> > This might seem a bit silly, but is there any chance we could add a 
> > comment at the end of pg_dump text output that says '-- End of dump'?
> 
> Sure --- while you're at it, put a "beginning of dump" at the start.
> 
> Is it worth adding the database name and/or other command-line
> parameters given to pg_dump?
> 
> One thing to think about is the difference between a bare pg_dump and a
> pg_dump/pg_restore sequence.  Should these always generate identical
> text output?  (They do as of CVS tip, I believe, though this was not
> always true before.)
> 
> Possibly this is all gilding the lily though...

I like an end-of-dump marker for folks who want to check if the dump got
truncated somehow.  I can see how to do that for text dumps, but what
about for tar or custom dumps?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tue, 30 Mar 2004 00:33:37 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pg_dump end comment
On Tue, 30 Mar 2004, Bruce Momjian wrote:

> Tom Lane wrote:
> > Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
> > > This might seem a bit silly, but is there any chance we could add a
> > > comment at the end of pg_dump text output that says '-- End of dump'?
> >
> > Sure --- while you're at it, put a "beginning of dump" at the start.
> >
> > Is it worth adding the database name and/or other command-line
> > parameters given to pg_dump?
> >
> > One thing to think about is the difference between a bare pg_dump and a
> > pg_dump/pg_restore sequence.  Should these always generate identical
> > text output?  (They do as of CVS tip, I believe, though this was not
> > always true before.)
> >
> > Possibly this is all gilding the lily though...
>
> I like an end-of-dump marker for folks who want to check if the dump got
> truncated somehow.  I can see how to do that for text dumps, but what
> about for tar or custom dumps?

Wouldn't it be more effective to test for non zero return status as this
handles -Fc cases, etc, which would be non-trivial to test.

Gavin

Tue, 30 Mar 2004 18:20:08 +1000 (EST)
Gavin Sherry <swm@linuxworld.com.au>
Re: pg_dump end comment
Gavin Sherry wrote:
> On Tue, 30 Mar 2004, Bruce Momjian wrote:
> 
> > Tom Lane wrote:
> > > Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
> > > > This might seem a bit silly, but is there any chance we could add a
> > > > comment at the end of pg_dump text output that says '-- End of dump'?
> > >
> > > Sure --- while you're at it, put a "beginning of dump" at the start.
> > >
> > > Is it worth adding the database name and/or other command-line
> > > parameters given to pg_dump?
> > >
> > > One thing to think about is the difference between a bare pg_dump and a
> > > pg_dump/pg_restore sequence.  Should these always generate identical
> > > text output?  (They do as of CVS tip, I believe, though this was not
> > > always true before.)
> > >
> > > Possibly this is all gilding the lily though...
> >
> > I like an end-of-dump marker for folks who want to check if the dump got
> > truncated somehow.  I can see how to do that for text dumps, but what
> > about for tar or custom dumps?
> 
> Wouldn't it be more effective to test for non zero return status as this
> handles -Fc cases, etc, which would be non-trivial to test.

Yes, they have to check for a proper exit from pg_dump, but there is
still a file sitting around after the dump, with no way to tell if it is
accurate.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tue, 30 Mar 2004 09:13:10 -0500 (EST)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: pg_dump end comment
Gavin Sherry <swm@linuxworld.com.au> writes:
> On Tue, 30 Mar 2004, Bruce Momjian wrote:
>> I like an end-of-dump marker for folks who want to check if the dump got
>> truncated somehow.  I can see how to do that for text dumps, but what
>> about for tar or custom dumps?

> Wouldn't it be more effective to test for non zero return status as this
> handles -Fc cases, etc, which would be non-trivial to test.

In the tar and custom archive formats, it's fairly obvious if the file
was truncated: you'll have table-of-contents entries that point past
the end of file.  The only case that really needs an endmarker is
textual scripts --- both straight from pg_dump, and produced by
pg_restore.

			regards, tom lane

Tue, 30 Mar 2004 09:26:13 -0500
Tom Lane <tgl@sss.pgh.pa.us>
Re: pg_dump end comment 
At 12:13 AM 31/03/2004, Bruce Momjian wrote:
>Yes, they have to check for a proper exit from pg_dump, but there is
>still a file sitting around after the dump, with no way to tell if it is
>accurate.

Why don't we write a hash into the header or footer. Then use something like:

     pg_restore --verify <dump-file>

if file integrity is the objective.



----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 03 5330 3172          |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                  |    --________--
PGP key available upon request,  |  /
and from pgp.mit.edu:11371       |/ 


Wed, 31 Mar 2004 11:26:31 +1000
Philip Warner <pjw@rhyme.com.au>
Re: pg_dump end comment
>>I like an end-of-dump marker for folks who want to check if the dump got
>>truncated somehow.  I can see how to do that for text dumps, but what
>>about for tar or custom dumps?
> 
> Wouldn't it be more effective to test for non zero return status as this
> handles -Fc cases, etc, which would be non-trivial to test.

That assumes you are there at the time of dump...

Chris


Wed, 31 Mar 2004 09:36:36 +0800
Christopher Kings-Lynne <chriskl@familyhealth.com.au>
Re: pg_dump end comment
On Wed, 31 Mar 2004, Philip Warner wrote:

> At 12:13 AM 31/03/2004, Bruce Momjian wrote:
> >Yes, they have to check for a proper exit from pg_dump, but there is
> >still a file sitting around after the dump, with no way to tell if it is
> >accurate.
> 
> Why don't we write a hash into the header or footer. Then use something like:
> 
>      pg_restore --verify <dump-file>
> 
> if file integrity is the objective.

I like this idea.  Nice to be able to check the md5 sig to make sure a 
backup is not corrupted OR short.


Wed, 31 Mar 2004 08:53:10 -0700 (MST)
"scott.marlowe" <scott.marlowe@ihs.com>
Re: pg_dump end comment
Tom Lane wrote:
> Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:
> > This might seem a bit silly, but is there any chance we could add a 
> > comment at the end of pg_dump text output that says '-- End of dump'?
> 
> Sure --- while you're at it, put a "beginning of dump" at the start.
> 
> Is it worth adding the database name and/or other command-line
> parameters given to pg_dump?
> 
> One thing to think about is the difference between a bare pg_dump and a
> pg_dump/pg_restore sequence.  Should these always generate identical
> text output?  (They do as of CVS tip, I believe, though this was not
> always true before.)

I have added code to print end-of-dump markers for pg_dump and
pg_dumpall:
	
	--
	-- PostgreSQL database dump complete
	--
	
	--
	-- PostgreSQL database cluster dump complete
	--

This mirrors the markers used at the start of the files.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Index: src/bin/pg_dump/pg_backup_archiver.c
===================================================================
RCS file: /cvsroot/pgsql-server/src/bin/pg_dump/pg_backup_archiver.c,v
retrieving revision 1.86
diff -c -c -r1.86 pg_backup_archiver.c
*** src/bin/pg_dump/pg_backup_archiver.c	22 Apr 2004 02:39:10 -0000	1.86
--- src/bin/pg_dump/pg_backup_archiver.c	19 May 2004 21:17:43 -0000
***************
*** 384,389 ****
--- 384,391 ----
  			AH->blobConnection = NULL;
  		}
  	}
+ 
+ 	ahprintf(AH, "--\n-- PostgreSQL database dump complete\n--\n\n");
  }
  
  /*
Index: src/bin/pg_dump/pg_dumpall.c
===================================================================
RCS file: /cvsroot/pgsql-server/src/bin/pg_dump/pg_dumpall.c,v
retrieving revision 1.33
diff -c -c -r1.33 pg_dumpall.c
*** src/bin/pg_dump/pg_dumpall.c	14 May 2004 17:04:47 -0000	1.33
--- src/bin/pg_dump/pg_dumpall.c	19 May 2004 21:17:43 -0000
***************
*** 242,247 ****
--- 242,252 ----
  	}
  
  	PQfinish(conn);
+ 
+ 	printf("--\n");
+ 	printf("-- PostgreSQL database cluster dump complete\n");
+ 	printf("--\n\n");
+ 
  	exit(0);
  }
Wed, 19 May 2004 17:21:12 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: [HACKERS] pg_dump end comment