Take the 2-minute tour ×

What do I have to do in Python to figure out which encoding?

share|improve this question
Related: stackoverflow.com/questions/196345/… –  eddflrs Feb 13 '11 at 22:33
Related: stackoverflow.com/q/1303243/64633 –  Rod Feb 13 '11 at 22:36
Unicode is not an encoding. –  ulidtko Feb 13 '11 at 22:42
More importantly, why should you care? –  Johnsyweb Feb 14 '11 at 3:30
add comment

6 Answers

up vote 58 down vote accepted

In Python 3, all strings are Unicode. There is a bytes type that holds raw bytes.

In Python 2, a string may be of type str or of type unicode. You can tell which using code something like this:

def whatisthis(s):
    if isinstance(s, str):
        print "ordinary string"
    elif isinstance(s, unicode):
        print "unicode string"
        print "not a string"
share|improve this answer
"all strings are Unicode"; needs to be re-worded: "all strings are sequences of Unicode characters." –  mkelley33 Feb 14 '11 at 1:55
Or maybe "all strings are in Unicode". –  ulidtko Feb 14 '11 at 6:46
Nope. All strings are just sequences of Unicode characters. "all strings are in Unicode" is also incorrect. –  mkelley33 Feb 14 '11 at 18:48
@mkelley33 we understand english so getting that precise on a simple question like this would over complicate things –  notbad.jpeg Jul 11 '12 at 19:01
@notbad.jpeg I understand, but the wording of the answer seems to suggest that there is a such thing as a "Unicode" string, which is wrong and different from what I said. The problem I have with the oversimplified wording is that is might serve to confuse an already oft misunderstood concept by improper use of the vocabulary. –  mkelley33 Jul 13 '12 at 14:47
add comment

Just do


One will say unicode, the other will say str.

You can handle them separately using isinstance, e.g.

if isinstance(s, str):
    print 's is a string object'
elif isinstance(s, unicode):
    print 's is a unicode object'

Or do you mean you have a str, and you are trying to figure out if it is encoded using ASCII or UTF-8 or something else?

In that case, try this:


if it raises an exception, the string is not 100% ASCII.

share|improve this answer
A downvote? Why? My answer is the same as Greg's but was first! –  Mikel Feb 13 '11 at 23:12
add comment

Unicode is not an encoding - to quote Kumar McMillan:

If ASCII, UTF-8, and other byte strings are "text" ...

...then Unicode is "text-ness";

it is the abstract form of text

Have a read of McMillan's Unicode In Python, Completely Demystified talk from PyCon 2008, it explains things a lot better than most of the related answers on Stack Overflow.

share|improve this answer
+1 for the helpful link. –  kojiro Jul 25 '12 at 23:29
add comment

If your code needs to be compatible with both Python 2 and Python 3, you can't directly use things like isinstance(s,bytes) or isinstance(s,unicode) without wrapping them in either try/except or a python version test, because bytes is undefined in Python 2 and unicode is undefined in Python 3.

There are some ugly workarounds. An extremely ugly one is to compare the name of the type, instead of comparing the type itself. Here's an example:

# convert bytes (python 3) or unicode (python 2) to str
if str(type(s)) == "<class 'bytes'>":
    # only possible in Python 3
    s = s.decode('ascii')  # or  s = str(s)[2:-1]
elif str(type(s)) == "<type 'unicode'>":
    # only possible in Python 2
    s = str(s)

An arguably slightly less ugly workaround is to check the Python version number, e.g.:

if sys.version_info >= (3,0,0):
    # for Python 3
    if isinstance(s, bytes):
    # for Python 2
    if isinstance(s, unicode):
        s = str(s)

Those are both unpythonic, and most of the time there's probably a better way.

share|improve this answer
The better way is probably to use six, and test against six.binary_type and six.text_type –  Ian Clelland Sep 26 '12 at 19:43
You can use type(s).__name__ to probe type names. –  Paulo Freitas Aug 26 '13 at 16:19
add comment

You could use Universal Encoding Detector, but be aware that it will just give you best guess, not the actual encoding, because it's impossible to know encoding of a string "abc" for example. You will need to get encoding information elsewhere, eg HTTP protocol uses Content-Type header for that.

share|improve this answer
add comment

In python 3.x all strings are sequences of Unicode characters. and doing the isinstance check for unicode should suffice.

With regards to python 2.x, Most people seem to be using an if statement that has two checks. one for str and one for unicode.

If you want to do it all with one statement though, you can do the following:

isinstance(x, basestring)
share|improve this answer
This is false. In Python 2.7 isinstance(u"x",basestring) returns True. –  PythonNut Jan 24 at 2:04
add comment

Your Answer


