
> No, what I envisioned was a standalone dumper that can produce dump output 
> without having a backend at all.  If this dumper knows about the various 
> binary formats, and knows how to get my data into a form I can then restore 
> reliably, I will be satisfied.  If it can be easily automated so much the 
> better.  Doing it table by table would be ok as well.

Unless it dumps binary representation of columns, a standalone dumper
would still need to load all the output function shared libs for custom types
(or not support custom types which would imho not be good).

Andreas



Mon, 8 Jul 2002 15:15:00 +0200
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at>
Re: (A) native Windows port
Zeugswetter Andreas SB SD wrote:
> 
> > No, what I envisioned was a standalone dumper that can produce dump output
> > without having a backend at all.  If this dumper knows about the various
> > binary formats, and knows how to get my data into a form I can then restore
> > reliably, I will be satisfied.  If it can be easily automated so much the
> > better.  Doing it table by table would be ok as well.
> 
> Unless it dumps binary representation of columns, a standalone dumper
> would still need to load all the output function shared libs for custom types
> (or not support custom types which would imho not be good).

And now we change the internal representation of NUMERIC to a short
integer array holding the number in base 10,000 and what exactly does
the standalone dumpster do with our data?

Another good example: let's add a field to some parsenode struct (was
there a release where this didn't happen?). This causes the NodeOut()
results to become a little different, which actually changes the textual
content of a likely toasted pg_rewrite attribute. Stored compressed and
sliced. I am quite a bit familiar with TOAST and the rewrite system.
Yet, someone has to help me a little to understand how we can do this
conversion in binary on the fly with an external tool. Especially where
this conversion results in different raw and compressed sizes of the
TOASTed attribute, which has to propagate up into the TOAST reference in
the main table ... not to speak of possible required block splits in the
toast table and index because of needing one more slice!


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #



Mon, 08 Jul 2002 15:20:46 -0400
Jan Wieck <JanWieck@Yahoo.com>
Re: (A) native Windows port
On Tue, 2002-07-09 at 00:20, Jan Wieck wrote:
> Zeugswetter Andreas SB SD wrote:
> > 
> > > No, what I envisioned was a standalone dumper that can produce dump output
> > > without having a backend at all.  If this dumper knows about the various
> > > binary formats, and knows how to get my data into a form I can then restore
> > > reliably, I will be satisfied.  If it can be easily automated so much the
> > > better.  Doing it table by table would be ok as well.
> > 
> > Unless it dumps binary representation of columns, a standalone dumper
> > would still need to load all the output function shared libs for custom types
> > (or not support custom types which would imho not be good).
> 
> And now we change the internal representation of NUMERIC to a short
> integer array holding the number in base 10,000 and what exactly does
> the standalone dumpster do with our data?
>
> Another good example: let's add a field to some parsenode struct (was
> there a release where this didn't happen?). This causes the NodeOut()
> results to become a little different, which actually changes the textual
> content of a likely toasted pg_rewrite attribute. Stored compressed and
> sliced. I am quite a bit familiar with TOAST and the rewrite system.
> Yet, someone has to help me a little to understand how we can do this
> conversion in binary on the fly with an external tool. Especially where
> this conversion results in different raw and compressed sizes of the
> TOASTed attribute, which has to propagate up into the TOAST reference in
> the main table ... not to speak of possible required block splits in the
> toast table and index because of needing one more slice!

This brings us back to my original proposal : this "external tool" needs
to be either a full postgres backend with added DUMP command or
something that can use a (possibly single-user) backend as either a 
library or, yes, a "backend" ;)

---------------------
Hannu





09 Jul 2002 00:24:13 +0500
Hannu Krosing <hannu@tm.ee>
Re: (A) native Windows port
On Monday 08 July 2002 03:20 pm, Jan Wieck wrote:
> Zeugswetter Andreas SB SD wrote:
> > Unless it dumps binary representation of columns, a standalone dumper
> > would still need to load all the output function shared libs for custom
> > types (or not support custom types which would imho not be good).

> And now we change the internal representation of NUMERIC to a short
> integer array holding the number in base 10,000 and what exactly does
> the standalone dumpster do with our data?

What does a standard dump/restore do then as well?  Is the restore process 
complicated by a rebuild of the function(s) involved in custom types?  This, 
IMHO, is a pathological case even for a standard dump/restore.  Someone doing 
this sort of thing is going to have more to do that a simple package upgrade.

> Another good example: let's add a field to some parsenode struct (was
> there a release where this didn't happen?). This causes the NodeOut()
> results to become a little different, which actually changes the textual
> content of a likely toasted pg_rewrite attribute. Stored compressed and
> sliced. I am quite a bit familiar with TOAST and the rewrite system.
> Yet, someone has to help me a little to understand how we can do this
> conversion in binary on the fly with an external tool. Especially where
> this conversion results in different raw and compressed sizes of the
> TOASTed attribute, which has to propagate up into the TOAST reference in
> the main table ... not to speak of possible required block splits in the
> toast table and index because of needing one more slice!

This is more difficult, certainly.  Martijn, how does pg_fsck handle such 
things now?

Again, this tool has utility outside upgrading.  And I'm talking about dumping 
the binary down to ASCII to be restored, not binary to binary on the fly.

This is the best dialog yet on the issue of upgrading.  Keep it coming! :-)
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Tue, 9 Jul 2002 11:19:09 -0400
Lamar Owen <lamar.owen@wgcr.org>
Re: (A) native Windows port
On Tue, 2002-07-09 at 17:19, Lamar Owen wrote:
> On Monday 08 July 2002 03:20 pm, Jan Wieck wrote:
> > Another good example: let's add a field to some parsenode struct (was
> > there a release where this didn't happen?). This causes the NodeOut()
> > results to become a little different, which actually changes the textual
> > content of a likely toasted pg_rewrite attribute. Stored compressed and
> > sliced. I am quite a bit familiar with TOAST and the rewrite system.
> > Yet, someone has to help me a little to understand how we can do this
> > conversion in binary on the fly with an external tool. Especially where
> > this conversion results in different raw and compressed sizes of the
> > TOASTed attribute, which has to propagate up into the TOAST reference in
> > the main table ... not to speak of possible required block splits in the
> > toast table and index because of needing one more slice!
> 
> This is more difficult, certainly.  Martijn, how does pg_fsck handle such 
> things now?
> 
> Again, this tool has utility outside upgrading.  And I'm talking about dumping 
> the binary down to ASCII to be restored, not binary to binary on the fly.

You seem to be talking about pg_dump + old backend, no ? ;)

For me it seems a given that you need old binary-reading code to read
old binary format data. The most convenient place to keep it is inside
an old backend. 

It may be possible to migrate simple user table data from old version to
new, but it gets complicated real fast for most other things, especially
for stuff in system tables.

---------------
Hannu


09 Jul 2002 18:57:54 +0200
Hannu Krosing <hannu@tm.ee>
Re: (A) native Windows port