
 Hi,

 I a little work with encodings (Japanese, Latin(s)) and I see that 
PG use non-standard encoding names.

 Why is here SJIS instead Shift-JIS, EUC_JP intead EUC-JP, 
Latin2 instead ISO-8859-2 ?

 It is not good for example for applications that output data to HTML and 
needs set correct meta-tags, for this is needful maintain in application
PostgreSQL's specific names and standard names and translate between these
...

 Comments?

 
			Karel


Wed, 21 Feb 2001 10:17:36 +0100 (CET)
Karel Zak <zakkr@zf.jcu.cz>
Encoding names
>  I a little work with encodings (Japanese, Latin(s)) and I see that 
> PG use non-standard encoding names.
> 
>  Why is here SJIS instead Shift-JIS, EUC_JP intead EUC-JP, 
> Latin2 instead ISO-8859-2 ?
> 
>  It is not good for example for applications that output data to HTML and 
> needs set correct meta-tags, for this is needful maintain in application
> PostgreSQL's specific names and standard names and translate between these
> ...
> 
>  Comments?

But HTML meta tags used to use their own encoding names such as
x-euc-jp, x-sjis....

Well, the reaons are:

1) shell does not like "-" (configure and some Unix commands in
   PostgreSQL accepts encoding names)
2) I don't like longer names

BTW, I and Thomas (and maybe others) are interested in implementing
CREATE CHRACATER SET staffs in SQL92/99. The encoding names might be
changed at that time...
--
Tatsuo Ishii

Wed, 21 Feb 2001 18:41:44 +0900
Tatsuo Ishii <t-ishii@sra.co.jp>
Re: Encoding names

> But HTML meta tags used to use their own encoding names such as
> x-euc-jp, x-sjis....

 Not sure, my mozilla understand "ISO-xxxx-x", "Shift-JIS" format too.
But it's irrelevant, important is that something like "Latin2" or "SJIS"
or "EUC_JP" are less standard names. And here aren't HTML only, but other
formats too (I-MODE, Wap, XML ...etc).

> Well, the reaons are:
> 
> 1) shell does not like "-" (configure and some Unix commands in
>    PostgreSQL accepts encoding names)
>
> 2) I don't like longer names

 Sorry, but both are great traverses and please never say "I don't like" 
if you talking about already existing standards, it's way to chaos.
 
 Sorry of this hard words, but I hope you understand me :-) 

> BTW, I and Thomas (and maybe others) are interested in implementing
> CREATE CHRACATER SET staffs in SQL92/99. The encoding names might be

 Well, I look forward.

			Karel


Wed, 21 Feb 2001 10:54:03 +0100 (CET)
Karel Zak <zakkr@zf.jcu.cz>
Re: Encoding names
> > But HTML meta tags used to use their own encoding names such as
> > x-euc-jp, x-sjis....
> 
>  Not sure, my mozilla understand "ISO-xxxx-x", "Shift-JIS" format too.
> But it's irrelevant, important is that something like "Latin2" or "SJIS"
> or "EUC_JP" are less standard names. And here aren't HTML only, but other
> formats too (I-MODE, Wap, XML ...etc).

They were introduced recently. If I remever correctly, when I started
to implemnet the multi-byte fucntionality, most of browsers did not
accept "Shift-JIS" as their meta tags.

> > Well, the reaons are:
> > 
> > 1) shell does not like "-" (configure and some Unix commands in
> >    PostgreSQL accepts encoding names)
> >
> > 2) I don't like longer names
> 
>  Sorry, but both are great traverses and please never say "I don't like" 
> if you talking about already existing standards, it's way to chaos.
>  
>  Sorry of this hard words, but I hope you understand me :-) 

Please understand there is no standard for charset/encoding names in
SQL92/99 itself. The SQL standard just says "you can import any
charset/encoding from anywhere if you can". Please correct me if I am
wrong.

However, I do not object to change encoding names if there are enough
agrees (and as long as the backward compatibilities are kept). 

> > BTW, I and Thomas (and maybe others) are interested in implementing
> > CREATE CHRACATER SET staffs in SQL92/99. The encoding names might be
> 
>  Well, I look forward.

Good.
--
Tatsuo Ishii

Wed, 21 Feb 2001 19:18:26 +0900
Tatsuo Ishii <t-ishii@sra.co.jp>
Re: Encoding names

On Wed, 21 Feb 2001, Tatsuo Ishii wrote:

> Please understand there is no standard for charset/encoding names in
> SQL92/99 itself. The SQL standard just says "you can import any
> charset/encoding from anywhere if you can". Please correct me if I am
> wrong.

In SQL standards not, but all probably known for example ISO names or 
some form for this.

> However, I do not object to change encoding names if there are enough
> agrees (and as long as the backward compatibilities are kept). 

 You not must change current names, you can add to pg_conv_tbl[] new lines
with names synonym for already existing encoding.

 An example:

 {LATIN1, "LATIN1",	0, latin12mic, mic2latin1, 0, 0}, 
 {LATIN1, "ISO-8859-1",	0, latin12mic, mic2latin1, 0, 0}, 

 And if you order this table by alphabet and in pg_char_to_encoding()
you use Knuth's binary search intead current seq. scannig by for() every
thing will faster and more nice. It's easy.

 What? :-)

		Karel



Wed, 21 Feb 2001 12:01:28 +0100 (CET)
Karel Zak <zakkr@zf.jcu.cz>
Re: Encoding names
>  You not must change current names, you can add to pg_conv_tbl[] new lines
> with names synonym for already existing encoding...
>  {LATIN1, "LATIN1",     0, latin12mic, mic2latin1, 0, 0},
>  {LATIN1, "ISO-8859-1", 0, latin12mic, mic2latin1, 0, 0},
>  And if you order this table by alphabet and in pg_char_to_encoding()
> you use Knuth's binary search intead current seq. scannig by for() every
> thing will faster and more nice. It's easy.

As you probably know, there is already a binary search algorithm coded
up for the date/time string lookups in utils/adt/datetime.c. Since that
lookup caches the last value (which could be done here too) most lookups
are immediate.

Are you proposing to make a change Karel, or just encouraging others? :)

                    - Thomas

Wed, 21 Feb 2001 13:38:45 +0000
Thomas Lockhart <lockhart@alumni.caltech.edu>
Re: Encoding names

On Wed, 21 Feb 2001, Thomas Lockhart wrote:

> >  You not must change current names, you can add to pg_conv_tbl[] new lines
> > with names synonym for already existing encoding...
> >  {LATIN1, "LATIN1",     0, latin12mic, mic2latin1, 0, 0},
> >  {LATIN1, "ISO-8859-1", 0, latin12mic, mic2latin1, 0, 0},
> >  And if you order this table by alphabet and in pg_char_to_encoding()
> > you use Knuth's binary search intead current seq. scannig by for() every
> > thing will faster and more nice. It's easy.
> 
> As you probably know, there is already a binary search algorithm coded
> up for the date/time string lookups in utils/adt/datetime.c. Since that
> lookup caches the last value (which could be done here too) most lookups
> are immediate.
> 
> Are you proposing to make a change Karel, or just encouraging others? :)
> 

 No problem for me. Do you want patch with this to tommorow breakfast?
IMHO it's acceptable for current 7.1 too, it's really small change.

 Or do it Tatsuo?

			Karel


Wed, 21 Feb 2001 15:13:24 +0100 (CET)
Karel Zak <zakkr@zf.jcu.cz>
Re: Encoding names
> > As you probably know, there is already a binary search algorithm coded
> > up for the date/time string lookups in utils/adt/datetime.c. Since that
> > lookup caches the last value (which could be done here too) most lookups
> > are immediate.
> > 
> > Are you proposing to make a change Karel, or just encouraging others? :)
> > 
> 
>  No problem for me. Do you want patch with this to tommorow breakfast?
> IMHO it's acceptable for current 7.1 too, it's really small change.
> 
>  Or do it Tatsuo?

Please go ahead. By the way, there is one more place you need to tweak
the encoding name table. Take a look at
interfaces/libpq/fe-connect.c. It's ugly to have simlilar tables in
two places, but I did not find better way to avoid to link huge
Unicode conversion tables in the frontend.
--
Tatsuo Ishii

Thu, 22 Feb 2001 00:05:07 +0900
Tatsuo Ishii <t-ishii@sra.co.jp>
Re: Encoding names

On Thu, 22 Feb 2001, Tatsuo Ishii wrote:

> > > As you probably know, there is already a binary search algorithm coded
> > > up for the date/time string lookups in utils/adt/datetime.c. Since that
> > > lookup caches the last value (which could be done here too) most lookups
> > > are immediate.
> > > 
> > > Are you proposing to make a change Karel, or just encouraging others? :)
> > > 
> > 
> >  No problem for me. Do you want patch with this to tommorow breakfast?
> > IMHO it's acceptable for current 7.1 too, it's really small change.
> > 
> >  Or do it Tatsuo?
> 
> Please go ahead. By the way, there is one more place you need to tweak
> the encoding name table. Take a look at
> interfaces/libpq/fe-connect.c. It's ugly to have simlilar tables in
> two places, but I did not find better way to avoid to link huge
> Unicode conversion tables in the frontend.

 Hmm, I see. It's realy a little ugly maintain two places with same
things. What this solution:

  * split (on backend) pg_conv_tbl[] into two tables:

	encstr2enc[] 	- for encoding names (strings) to encode 'id'.
			  This table will sort by alphabet.   
			
	pg_conv_tbl[]	- table with encoding 'id' and with encoding routines.
			  This table will order by encoding 'id' and this
			  order allows found relevant routines, an example:

	pg_conv_tbl[ LATIN1 ]


 This solution allows to use and share on libpq and backend one encstr2enc[] 
table and basic functions those works with this table -- like
pg_char_to_encoding().

 May be will better define for encoding 'id' separate enum datetype instead
current mistake-able '#define'. 

			Karel


Wed, 21 Feb 2001 16:42:20 +0100 (CET)
Karel Zak <zakkr@zf.jcu.cz>
Re: Encoding names