
> Binary-encoded numeric values don't go through
> encoding conversion.  Text strings currently do.  The only place where I
> had any difficulty deciding what a particular datatype should do is with
> the 1-byte "char" type, which has a foot in both camps.  I decided to
> treat it as an unconverted single byte (but am willing to listen to
> argument if anyone thinks differently).

One concern I would have is how to get at the raw unconverted text data
without changing the client encoding. I do not really see an advantage in
providing the same thing for text and binary format if it is now 
a per column thing. I would do text unconverted if binary format is selected
for that column.

Did COPY BINARY convert text ?

Andreas

Wed, 14 May 2003 11:09:44 +0200
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at>
Re: GUC and postgresql.conf docs 
"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:
>> Binary-encoded numeric values don't go through
>> encoding conversion.  Text strings currently do.  The only place where I
>> had any difficulty deciding what a particular datatype should do is with
>> the 1-byte "char" type, which has a foot in both camps.  I decided to
>> treat it as an unconverted single byte (but am willing to listen to
>> argument if anyone thinks differently).

> One concern I would have is how to get at the raw unconverted text data
> without changing the client encoding. I do not really see an advantage in
> providing the same thing for text and binary format if it is now 
> a per column thing. I would do text unconverted if binary format is
> selected for that column.

That's a good point: as things currently stand in CVS tip, there's
really no difference between text and binary output of textual
datatypes.  Perhaps that's how it should be, but one could make an
argument that we're missing a chance to expose possibly-useful
functionality.

> Did COPY BINARY convert text ?

COPY BINARY currently does the same conversions as binary I/O to the
client.  This is arguably reasonable when doing COPY to/from the
frontend, but it's probably not very reasonable when doing COPY to/from
a file in the server's filesystem --- there's no obvious reason why that
should be affected by client_encoding.  And yet, I'm not sure the cases
should be different either.  Wouldn't it be a bad thing if a COPY dump
made through psql's \copy didn't reload correctly through "COPY FROM
file"?

We could sidestep that issue if binary I/O for text was in server
encoding in all cases.

			regards, tom lane

Wed, 14 May 2003 11:35:06 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Client encoding conversion for binary data (was Re: GUC and
	postgresql.conf docs)
Tom Lane writes:

> That's a good point: as things currently stand in CVS tip, there's
> really no difference between text and binary output of textual
> datatypes.  Perhaps that's how it should be, but one could make an
> argument that we're missing a chance to expose possibly-useful
> functionality.

Let's look at this in a broader context.  Depending on the data type,
there may be zero, one, or more useful text formats and zero, one, or more
useful binary formats.  The general rule ought to be that text formats are
subject to character set conversion, and the binary formats are not.

For example, the type timestamp might have:

text format "ISO"
text format "German"
binary format "struct tm"
binary format "time_t"

and possibly others.  I would expect the format two to be subject to
character set conversion, but the latter two not.

That same rule applied to character types would say that the "normal" text
format is subject to character set conversion (of course), and any other
text format (whatever that would be) would also be.  Any binary format for
character types would not be subject to character set conversion.  But
that does not say what would be in that binary format.  It could be the
internal server encoding representation or mule internal code or the data
preconverted to the client encoding outside the automatic mechanisms or
anything else.  Unless someone can come up with a binary representation
that would be genuinely useful, the simplest answer would be that
character types don't have one and you have to use the text format.

-- 
Peter Eisentraut   peter_e@gmx.net


Thu, 15 May 2003 17:39:37 +0200 (CEST)
Peter Eisentraut <peter_e@gmx.net>
Re: Client encoding conversion for binary data (was Re: 
Peter Eisentraut <peter_e@gmx.net> writes:
> That same rule applied to character types would say that the "normal" text
> format is subject to character set conversion (of course), and any other
> text format (whatever that would be) would also be.  Any binary format for
> character types would not be subject to character set conversion.  But
> that does not say what would be in that binary format.  It could be the
> internal server encoding representation or mule internal code or the data
> preconverted to the client encoding outside the automatic mechanisms or
> anything else.

True.

> Unless someone can come up with a binary representation
> that would be genuinely useful, the simplest answer would be that
> character types don't have one and you have to use the text format.

You're prejudging the question at hand, which is whether or not access
to the server-encoding representation is useful.  I'm inclined to think
that it is, particularly for backup purposes (no need to worry about
lossage from character set conversions).  Of course one can set
client_encoding equal to server_encoding to get the same effect, but
that doesn't seem to be an argument that it's not useful.

			regards, tom lane

Thu, 15 May 2003 11:48:50 -0400
Tom Lane <tgl@sss.pgh.pa.us>
Re: Client encoding conversion for binary data (was Re: GUC and
	postgresql.conf docs)