Beefy Boxes and Bandwidth Generously Provided by pair Networks kudra
Just another Perl shrine

Re: Comparing strings at grapheme level

by halley (Prior)
on Jan 09, 2008 at 14:51 UTC ( #661367=note: print w/ replies, xml ) Need Help??

in reply to Comparing strings at grapheme level

I just did a google search and it looks like this is fairly authoritative, but still not quite helpful in your situation.


    It would seem that if the normalization functions worked to that spec, then they would always produce the same output order. However, in my limited experience I've never seen any real-world cases where you'd want or need multiple COMBINING characters... not saying it doesn't exist but the Unicode docs don't obviously mention it specifically.

    I would say that if you're straying outside the defined normalization routines, that you gotta do something stable and predictable, like sort your multiple COMBINING elements into numerical codepoint order. (Order of these should never matter for the grapheme, right?)

    Update: This blog says ordering does matter, and that they should be sorted by class of combining element. The ordering of combining elements in the same class may or may not need to be maintained, which is the issue you're running into. Pretty annoying if there's not a standard library implementation of all this.

    My perlish hat says that to compare two graphemes, you'd have to do a hash/set of included characters, then compare the hashes for equivalence. If those match, and the order of combining classes is the same, it's the same grapheme.


  • Comment on Re: Comparing strings at grapheme level
    Re^2: Comparing strings at grapheme level
    by moritz (Cardinal) on Jan 10, 2008 at 10:02 UTC
      Thanks for your helpful post.

      At the moment I have no real Unicode problems (but I rather try to understand the concepts). I'll keep that in mind when I start do some real work with Unicode.

    Log In?

    What's my password?
    Create A New User
    Node Status?
    node history
    Node Type: note [id://661367]
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others taking refuge in the Monastery: (7)
    As of 2014-03-15 16:36 GMT
    Find Nodes?
      Voting Booth?

      Have you used a cryptocurrency?

      Results (318 votes), past polls