> -----Original Message-----
> From: pgsql-hackers-owner@postgresql.org 
> [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of 
> Dennis Bjorklund
> Sent: Saturday, August 07, 2004 10:48 PM
> To: Takehiko Abe
> Cc: pgsql-hackers@postgresql.org
> Subject: Re: [PATCHES] [HACKERS] UNICODE characters above 0x10000
> 
> On Sat, 7 Aug 2004, Takehiko Abe wrote:
> 
> It looked like you sent the last mail only to me and not the 
> list. I assume it was a misstake and I send the reply to both.
> 
> > > Is there a specific reason you want to restrict it to 24 bits?
> > 
> > ISO 10646 is said to have removed its private use codepoints outside 
> > of the Unicode 0 - 10FFFF range to ensure the compatibility with Unicode.
> > 
> > see Section C.2 and C.3 of Unicode 4.0 Appendix C 
> "Relationship to ISO
> > 10646": <http://www.unicode.org/versions/Unicode4.0.0/appC.pdf>.
> 
> The one and only reason for allowing 31 bit is that it's 
> defined by iso 10646. In practice there is probably no one 
> that uses the upper part of
> 10646 so not supporting it will most likely not hurt anyone.
>                                                               
>                   
> I'm happy either way so I will put my voice on letting PG use 
> unicode (not ISO 10646) and restrict it to 24 bits. By the 
> time someone wants (if ever) iso 10646 we probably have 
> support for different charsets and can easily handle both at 
> the same time.
> 

Point taken. 
Since we're supporting UTF8, and not ISO 10646.

Now, is it really 24 bits tho? 
Afaict, it's really 21 (0 - 10FFFF or 0 - xxx10000 11111111 11111111)

This would require that we suport 4 byte sequences
(11110100 10001111 10111111 10111111 = 10FFFF)

> --
> /Dennis Björklund
> 
> 
> ---------------------------(end of 
> broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings
> 
> 


Regards,

John Hansen

Sat, 7 Aug 2004 23:14:24 +1000
"John Hansen" <john@geeknet.com.au>
Re: [PATCHES] UNICODE characters above 0x10000
On Sat, 7 Aug 2004, John Hansen wrote:

> Now, is it really 24 bits tho? 
> Afaict, it's really 21 (0 - 10FFFF or 0 - xxx10000 11111111 11111111)

Yes, up to 0x10ffff should be enough.

The 24 is not really important, this is all about what utf-8 strings to 
accept as input. The strings are stored as utf-8 strings and when 
processed inside pg it uses wchar_t that is 32 bit (on some systems at 
least). By restricting the utf-8 input to unicode we can in the future 
store each character as 3 bytes if we want.

--
/Dennis Bjï¿½rklund


Sat, 7 Aug 2004 15:22:34 +0200 (CEST)
Dennis Bjorklund <db@zigo.dhs.org>
Re: [PATCHES] UNICODE characters above 0x10000