title: UnicodeDecodeError : invalid continuation byte in reading filenames in Python
body: <p>I went through all the various questions that were already posted here and tried the solutions mentioned but just couldn't get my code to work. </p>
  
  <p>I am trying to read the filenames within directories using python but keep running into the following error.</p>
  
  <blockquote>
    <p>UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 907: invalid continuation byte</p>
  </blockquote>
  
  <p><strong>Code</strong></p>
  
  <pre><code>from __future__ import print_function
  import sklearn.datasets
  import nltk
  
  dataset = sklearn.datasets.load_files('data/', shuffle='False')
  categories = dataset.target_names
  
  from os import listdir
  from os.path import isfile, join
  
  for c in categories:
      directory_path = 'data/'+c
      onlyfiles = [f for f in listdir(directory_path) if isfile(join(directory_path, f))]
      print ("Level 1 Intent : ", c)
      print ("---------------------------------------")
      for file_name in onlyfiles:
          file_path = directory_path+'/'+file_name
          with open(file_path) as f:
              first_line = f.readline()
              print("Level 2 Intent for ", c, " : ", first_line)
  
              stopwords = nltk.corpus.stopwords.words('english')
              from nltk.stem.snowball import SnowballStemmer
              stemmer = SnowballStemmer("english")
  </code></pre>
  
  <p>Though I get the output of the filenames printed as required, I run into this error message. </p>
  
  <p><strong>Sample Output</strong></p>
  
  <pre><code>...
  Level 2 Intent for  Presidents  :  Gerald Ford
  Level 1 Intent :  Scientists
  ---------------------------------------
  Level 2 Intent for  Scientists  :  Alessandro_Volta
  Level 2 Intent for  Scientists  :  Michael_Faraday
  Level 2 Intent for  Scientists  :  James_Watt
  Level 2 Intent for  Scientists  :  Nikola_Tesla
  ...
  </code></pre>
  
  <p><strong>Sample Filenames</strong></p>
  
  <pre><code>1.clean
  2.clean
  S08_set3_a1.txt.clean
  S08_set3_a8.txt.clean
  </code></pre>
  
  <p>Any suggestions in this regard will be highly appreciated.</p>