title: pyspark: TypeError: IntegerType can not accept object in type <type 'unicode'>
body: <p>programming with pyspark on a Spark cluster,
  the data is large and in pieces so can not be loaded into the memory or check the sanity of the data easily</p>
  
  <p>basically it looks like</p>
  
  <pre><code>af.b Current%20events 1 996
  af.b Kategorie:Musiek 1 4468
  af.b Spesiaal:RecentChangesLinked/Gebruikerbespreking:Freakazoid 1 5209
  af.b Spesiaal:RecentChangesLinked/Sir_Arthur_Conan_Doyle 1 5214
  </code></pre>
  
  <p>wikipedia data:</p>
  
  <p>I read it from aws S3 and then try to construct spark Dataframe with the following python code in pyspark intepreter:</p>
  
  <pre><code>parts = data.map(lambda l: l.split())
  wikis = parts.map(lambda p: (p[0], p[1],p[2],p[3]))
  
  
  fields = [StructField("project", StringType(), True),
  StructField("title", StringType(), True),
  StructField("count", IntegerType(), True),
  StructField("byte_size", StringType(), True)] 
  
  schema = StructType(fields) 
  
  df = sqlContext.createDataFrame(wikis, schema)
  </code></pre>
  
  <p>all look fine, only createDataFrame gives me error</p>
  
  <pre><code>Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/usr/lib/spark/python/pyspark/sql/context.py", line 404, in   createDataFrame
  rdd, schema = self._createFromRDD(data, schema, samplingRatio)
  File "/usr/lib/spark/python/pyspark/sql/context.py", line 298, in _createFromRDD
  _verify_type(row, schema)
  File "/usr/lib/spark/python/pyspark/sql/types.py", line 1152, in _verify_type
  _verify_type(v, f.dataType)
  File "/usr/lib/spark/python/pyspark/sql/types.py", line 1136, in _verify_type
  raise TypeError("%s can not accept object in type %s" % (dataType, type(obj)))
  TypeError: IntegerType can not accept object in type &lt;type 'unicode'&gt;
  </code></pre>
  
  <p>why I can not set the third column which should be count to IntegerType ?
  How can I solve this ?</p>