Uh oh -

    on  one side I'm a happy camper. pg_dump already ignored pg_*
    tables and since the TOAST  tables  are  named  pg_toast_...,
    nothing  to  be done. I loaded my test DB with TOAST entries,
    dumped and restored it.  Anything is there, works  perfectly.

    But  then  I added the ALTER TABLE for unlimited rewrite rule
    size to initdb and the problems started. I can create a table
    with  500+  attributes.  Also  I can create a view on it (the
    rules size is 170K - whow). Anything works pretty well,  just
    a pg_dump output is garbage.

    Seems  the  dynamic  string buffers used in pg_dump aren't as
    bullet proof as they  should.   I'm  still  busy  with  other
    things, so can someone please take a look on it?  Attached is
    an SQL script that creates the table  and  the  view  that  I
    cannot dump.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #
Wed, 5 Jul 2000 22:01:49 +0200 (MEST)
JanWieck@t-online.de (Jan Wieck)
2nd update on TOAST
JanWieck@t-online.de (Jan Wieck) writes:
>     size to initdb and the problems started. I can create a table
>     with  500+  attributes.  Also  I can create a view on it (the
>     rules size is 170K - whow). Anything works pretty well,  just
>     a pg_dump output is garbage.

Are you using current sources?  Bruce committed Philip Warner's
pg_dump rewrite a day or so ago (which I thought was way premature,
but anyway...).  Just want to know which pg_dump we're talking about.

			regards, tom lane

Wed, 05 Jul 2000 17:18:49 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: 2nd update on TOAST 
At 22:01 5/07/00 +0200, Jan Wieck wrote:
>
>    Seems  the  dynamic  string buffers used in pg_dump aren't as
>    bullet proof as they  should.   I'm  still  busy  with  other
>    things, so can someone please take a look on it?  Attached is
>    an SQL script that creates the table  and  the  view  that  I
>    cannot dump.
>

If the TOAST patch on the FTP site the most recent?


----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.C.N. 008 659 498)             |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/

Thu, 06 Jul 2000 12:37:39 +1000
Philip Warner <pjw@rhyme.com.au>
Re: 2nd update on TOAST
At 17:18 5/07/00 -0400, Tom Lane wrote:
>JanWieck@t-online.de (Jan Wieck) writes:
>>     size to initdb and the problems started. I can create a table
>>     with  500+  attributes.  Also  I can create a view on it (the
>>     rules size is 170K - whow). Anything works pretty well,  just
>>     a pg_dump output is garbage.
>
>Are you using current sources?  Bruce committed Philip Warner's
>pg_dump rewrite a day or so ago (which I thought was way premature,
>but anyway...).  Just want to know which pg_dump we're talking about.

It's good that any bugs had a chnace to come out ASAP.

However, I'd be interested to know what actually happens: I do recall that
pg_dump (both versions) have seomething like '#define COPY_BUFFER_SIZE
8192'; if I were to start looking, that's where I'd go first.


----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.C.N. 008 659 498)             |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/

Thu, 06 Jul 2000 12:50:06 +1000
Philip Warner <pjw@rhyme.com.au>
Re: 2nd update on TOAST 
At 12:50 6/07/00 +1000, Philip Warner wrote:
>>
>>Are you using current sources?  Bruce committed Philip Warner's
>>pg_dump rewrite a day or so ago (which I thought was way premature,
>>but anyway...).  Just want to know which pg_dump we're talking about.
>
>It's good that any bugs had a chnace to come out ASAP.
>
>However, I'd be interested to know what actually happens: I do recall that
>pg_dump (both versions) have seomething like '#define COPY_BUFFER_SIZE
>8192'; if I were to start looking, that's where I'd go first.
>

Further to this, looking at the code, it now uses 'archputs', which
replaces 'fputs', to output the copy buffer (assuming the error is in the
copy, not while dumping the definitions); if PQgetline fills the entire
buffer (no trailing \0), then I could imagine both would croak. The
simplest test would be to tell PQgetline that the buffer size if 1 byte
smaller, and see if it fixes the problem. 

I'd be interested to hear from someone...at least to know a little more of
the circumstaces of the error.



----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.C.N. 008 659 498)             |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/

Thu, 06 Jul 2000 13:15:55 +1000
Philip Warner <pjw@rhyme.com.au>
Re: 2nd update on TOAST 
At 13:15 6/07/00 +1000, Philip Warner wrote:
>At 12:50 6/07/00 +1000, Philip Warner wrote:
>>>
>>>Are you using current sources?  Bruce committed Philip Warner's
>>>pg_dump rewrite a day or so ago (which I thought was way premature,
>>>but anyway...).  Just want to know which pg_dump we're talking about.
>>

OK, I've built from the latest CVS, run initdb etc, created a new DB,
created the megatable & megaview, and done a pg_dump. It works, except that
I introduced a but that caused the view to be dumped as a table as well. I
will keep looking, and submit a patch shortly. In the mean time, if anyone
can reproduce Jan's problem, that would help...


----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.C.N. 008 659 498)             |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/

Thu, 06 Jul 2000 15:28:49 +1000
Philip Warner <pjw@rhyme.com.au>
Re: 2nd update on TOAST 
Philip Warner wrote:
> At 22:01 5/07/00 +0200, Jan Wieck wrote:
> >
> >    Seems  the  dynamic  string buffers used in pg_dump aren't as
> >    bullet proof as they  should.   I'm  still  busy  with  other
> >    things, so can someone please take a look on it?  Attached is
> >    an SQL script that creates the table  and  the  view  that  I
> >    cannot dump.
> >
> 
> If the TOAST patch on the FTP site the most recent?

    No. It's all in the current CVS tree. I removed that patch
    already.


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #


Thu, 6 Jul 2000 11:09:09 +0200 (MEST)
JanWieck@t-online.de (Jan Wieck)
Re: 2nd update on TOAST
> JanWieck@t-online.de (Jan Wieck) writes:
> >     size to initdb and the problems started. I can create a table
> >     with  500+  attributes.  Also  I can create a view on it (the
> >     rules size is 170K - whow). Anything works pretty well,  just
> >     a pg_dump output is garbage.
> 
> Are you using current sources?  Bruce committed Philip Warner's
> pg_dump rewrite a day or so ago (which I thought was way premature,
> but anyway...).  Just want to know which pg_dump we're talking about.

I committed pg_dump so people could see his changes and start making
additions.  I can then have him supply incremental patches.  Keeping
stuff out of the tree usually causes the author to get frustrated.

Of course, if it comes in too quickly, it can cause chaos if he needs to
make major changes.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Thu, 6 Jul 2000 16:35:34 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us>
Re: 2nd update on TOAST