title: Encoding to UTF-8 files in hadoop
body: <p>I'm writing a MapReduce program in order to clean some files stored in HDFS, for that i have to encode all files in UTF-8, i tried to encode the Text value in my mapper but i still have errors in my result file. </p>
  
  <pre><code>if(encoding.compareTo("UTF-8")!=0){
              final Charset fromCharset = Charset.forName(encoding);
              final Charset toCharset = Charset.forName("UTF-8");
              String fixed = new String(value.toString().getBytes(fromCharset), toCharset); 
              result= new String(fixed);
  </code></pre>
  
  <p>I also custom the LineReader in order to encode the bytes readed into UTF-8 before that it's stored in Text Object.</p>
  
  <pre><code> //buffer contain the data readed in a line of the file
   String s = new String(buffer, startPosn, appendLength);
              byte ptext[] = Charset.forName("UTF-8").encode(s).array();              
              str.append(ptext, 0, ptext.length);
  </code></pre>
  
  <p>Can you help me please ! </p>