title: Unicode: Automatically detect high characters and convert them to surrogate pairs
body: <p>I want to write a program in python that writes a string to a file in a manner that a java application can read it. This java application reads the string in modified UTF-8 format (see <a href="https://stackoverflow.com/questions/7921016/what-does-it-mean-to-say-java-modified-utf-8-encoding">this question</a>). It basically means that all unicode characters have to be written as surrogate characters aka UTF-16BE. Is there a graceful, effective method that can do this.</p>
  
  <p>I have tried the following to do so:</p>
  
  <pre><code>import re;
  o_string = b''
  for splitter in re.split("([\U00010000-\U0010FFFF])", w_string):
      if len(splitter) == 1 and ord(splitter) &gt; 0xFFFF:
          o_string += splitter.encode("UTF-16")
      else:
          o_string += splitter.decode("UTF-8")
  print(o_string)
  </code></pre>
  
  <p>But this seems so bulky (also it uses regex which seems to be en easy to dodge overhead). Can someone propose a better solution to this?</p>