title: Spark Python map function: Error in encoding utf-8
body: <p>I've been trying to process a textfile with Python. My textfile is in the native language of Vietnamese people, which use the UTF-8. After I use the map function, the output seems to be out of format. I've separated the code and output it step by step. I notice that the encoding went wrong after the map word: (word, 1). To be more precise, upto output7.txt, the text was: 
  Đức
  đã
  ngã
  gục</p>
  
  <p>However, at output 8, the encoding went wrong:</p>
  
  <p>(u'\u0110\u1ee9c', 1)
  (u'\u0111\xe3', 1)
  (u'ng\xe3', 1)
  (u'g\u1ee5c', 1)      </p>
  
  <p>it's supposed to be 
  (Đức, 1)
  (đã, 1)
  (ngã, 1)
  (gục, 1)</p>
  
  <p>I've tried to fix this bug for 5 hours, but haven't found anything really useful. Can anyone tells me why the Map function wrecked everything, while the similar FlatMap function works fine?</p>
  
  <p>Thank you. Below is my source code. </p>
  
  <pre><code>&gt; #!/usr/bin/python
  # -*- coding: utf8 -*-
  from pyspark import SparkContext, SparkConf
  import os, sys
  import codecs
  
  conf =SparkConf().setAppName("wordcount").setMaster("local")
  sc = SparkContext(conf=conf)
  
  reload(sys)
  sys.setdefaultencoding('utf-8')
  
  text_file = sc.textFile("outputtest/*",use_unicode=False)
  dict_file = sc.textFile("keyword");
  
  text_file.saveAsTextFile("Output6.txt")
  
  counts = text_file.flatMap(lambda line: line.split(" "))
  counts.saveAsTextFile("Output7.txt")
  
  counts = counts.map(lambda word: (word.decode("utf-8"), 1)) 
  counts.saveAsTextFile("Output8.txt")
  
  counts= counts.reduceByKey(lambda a, b: a + b)
  dicts = dict_file.flatMap(lambda line: line.split(", ")) \
                   .map(lambda word: (word.replace("'","").replace(" ","_"))) 
  
  keyword = dicts.join(counts);                
  
  counts.saveAsTextFile("Output9.txt")
  </code></pre>