title: Python general unicode
body: <p>I'm having problems understanding unicode in python 2.7.2, so I tried some tests in idle. Two things are marked 'not sure'.  Please tell me why they fail. As to the other items, please tell me if my comment is accurate.</p>
  
  <pre><code>&gt;&gt;&gt; s
  'Don\x92t '  # s is a string
  &gt;&gt;&gt; u
  u'Don\u2019t '  # u is a unicode object
  &gt;&gt;&gt; type(u)     # confirm u is unicode
  &lt;type 'unicode'&gt;
  &gt;&gt;&gt; type(s)     # confirm s is string
  &lt;type 'str'&gt;
  &gt;&gt;&gt; type(s) == 'str' # wrong way to test
  False
  &gt;&gt;&gt; isinstance(s, str)  # right way to test
  True
  &gt;&gt;&gt; print s
  Don’t       # works because idle can handle strings
  &gt;&gt;&gt; print u
  Don’t       # works because idle can handle unicode
  &gt;&gt;&gt; open('9', 'w').write(s.encode('utf8')) #encode takes unicode, but s is a string,
                                              # so this fails
  Traceback (most recent call last):
    File "&lt;pyshell#28&gt;", line 1, in &lt;module&gt;
      open('9', 'w').write(s.encode('utf8'))
  UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position 3: ordinal not in range(128)
  &gt;&gt;&gt; open('9', 'w').write(s) # write can write strings
  &gt;&gt;&gt; open('9', 'w').write(u) # write can't write unicode
  
  Traceback (most recent call last):
    File "&lt;pyshell#30&gt;", line 1, in &lt;module&gt;
      open('9', 'w').write(u)
  UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 3: ordinal not in range(128)
  &gt;&gt;&gt; open('9', 'w').write(u.encode('utf8'))  # encode turns unicode to string, which write can handle
  &gt;&gt;&gt; open('9', 'w').write(s.decode('utf8'))  # decode turns string to unicode, which write can't handle
  
  Traceback (most recent call last):
    File "&lt;pyshell#32&gt;", line 1, in &lt;module&gt;
      open('9', 'w').write(s.decode('utf8'))
    File "C:\program files\Python27\lib\encodings\utf_8.py", line 16, in decode
      return codecs.utf_8_decode(input, errors, True)
  UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 3: invalid start byte
  &gt;&gt;&gt; e = '{}, {}'.format(s, u) # fails becase ''.format is string, while u is unicode
  
  Traceback (most recent call last):
    File "&lt;pyshell#33&gt;", line 1, in &lt;module&gt;
      e = '{}, {}'.format(s, u)
  UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 3: ordinal not in range(128)
  &gt;&gt;&gt; e = '{}, {}'.format(s, u.encode('utf8')) # works because u.encode is a string
  &gt;&gt;&gt; e = u'{}, {}'.format(s, u) # not sure
  
  Traceback (most recent call last):
    File "&lt;pyshell#36&gt;", line 1, in &lt;module&gt;
      e = u'{}, {}'.format(s, u)
  UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position 3: ordinal not in range(128)
  &gt;&gt;&gt; e = u'{}, {}'.format(s.decode('utf8'), u) # not sure
  
  Traceback (most recent call last):
    File "&lt;pyshell#55&gt;", line 1, in &lt;module&gt;
      e = u'{}, {}'.format(s.decode('utf8'), u)
    File "C:\program files\Python27\lib\encodings\utf_8.py", line 16, in decode
      return codecs.utf_8_decode(input, errors, True)
  UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 3: invalid start byte
  
  &gt;&gt;&gt; e = '\n'.join([s, u]) # wants strings, but u is unicode
  
  Traceback (most recent call last):
    File "&lt;pyshell#37&gt;", line 1, in &lt;module&gt;
      e = '\n'.join([s, u])
  UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position 3: ordinal not in range(128)
  &gt;&gt;&gt; e = '\n'.join([s, u.encode('utf8')]) # u.encode is now a string
  </code></pre>