title: spark to hive data types
body: <p>is there a way, how to convert an input string field into orc table with column specified as varchar(xx) in sparksql select query? Or I have to use some workaroud? I'm using Spark 1.6.</p>
  
  <p>I found on Cloudera forum, Spark does not care about length, it saves the value as string with no size limit. </p>
  
  <p>The table is inserted into Hive OK, but I'm little bit worried about data quality.     </p>
  
  <pre><code>temp_table = sqlContext.table(ext)
  df = temp_table.select(temp_dable.day.cast('string'))
  </code></pre>
  
  <p>I would like to see something like that :)))</p>
  
  <pre><code> df = temp_table.select(temp_dable.day.cast('varchar(100)'))
  </code></pre>
  
  <p>Edit: </p>
  
  <pre><code>df.write.partitionBy(part).mode('overwrite').insertInto(int)
  </code></pre>
  
  <p>Table I'm inserting into is saved as an ORC file (the line above probably should have .format('orc')). </p>
  
  <p>I found <a href="https://community.hortonworks.com/questions/48260/hive-string-vs-varchar-performance.html" rel="nofollow noreferrer">here</a>, that If I specify a column as a varchar(xx) type, than the input string will be cutoff to the xx length. </p>
  
  <p>Thx</p>