Hannu Krosing wrote:
> IIRC, the charset transformations are done as a separate step in the
> wire protocol _before_ any parser has chance transform or not.

Yep.  My point is that this is wrong.  I think of XML the same way I
think of a zip file contains a text document.  Postgres does not unzip a
text file to change the char encoding any more than it should parse an
XML document and change the encoding unless this is the specific intent
of the user for a specific purpose.  

Validation alone does not qualify as a reason because a XML parser
(xerces) can do validation server-side without mucking with document.
Postgres need only be aware of the fact that the data is XML and should
be validated.

If postgres needs to be aware of internal document contents (for
indexing, for example), XSLT can be used for that purpose.  

Regards,
Merlin


Mon, 12 Jan 2004 12:56:32 -0500
"Merlin Moncure" <merlin.moncure@rcsonline.com>
Re: Encoding problems in PostgreSQL with XML data
Merlin Moncure kirjutas E, 12.01.2004 kell 19:56:
> Hannu Krosing wrote:
> > IIRC, the charset transformations are done as a separate step in the
> > wire protocol _before_ any parser has chance transform or not.
> 
> Yep.  My point is that this is wrong.  

Of course :)

It seems to be a quick hack somebody implemented in need, and doing it
as a separate step was suely the easiest way to get it working.

I hope that real as-needed-column-by-column translation will be used
with bound argument queries.

It also seems possible to delegate the encoding changes to after the
query is parsed, but this will never work for EBCDIC and other funny
encodings (like rot13 ;). 

for these we need to define the actual SQL statement encoding on-wire to
be always ASCII.

---------------
Hannu


Tue, 13 Jan 2004 14:42:41 +0200
Hannu Krosing <hannu@tm.ee>
Re: Encoding problems in PostgreSQL with XML data
> Merlin Moncure kirjutas E, 12.01.2004 kell 19:56:
> > Hannu Krosing wrote:
> > > IIRC, the charset transformations are done as a separate step in the
> > > wire protocol _before_ any parser has chance transform or not.
> > 
> > Yep.  My point is that this is wrong.  
> 
> Of course :)

We need this because our parser cannot handle some encodings including
UCS, Shift-JIS and Big5 (we currently do not allow UCS as a client
side encoding, but this is another story).

> It seems to be a quick hack somebody implemented in need, and doing it
> as a separate step was suely the easiest way to get it working.
> 
> I hope that real as-needed-column-by-column translation will be used
> with bound argument queries.

IMO this does not help our parser's limitation neither.

> It also seems possible to delegate the encoding changes to after the
> query is parsed, but this will never work for EBCDIC and other funny
> encodings (like rot13 ;). 
> 
> for these we need to define the actual SQL statement encoding on-wire to
> be always ASCII.

Wed, 21 Jan 2004 13:51:52 +0900 (JST)
Tatsuo Ishii <t-ishii@sra.co.jp>
Re: Encoding problems in PostgreSQL with XML data