title: creating hive table using gcloud dataproc not working for unicode delimiter
body: <p>I need to create a hive table on a unicode delimited file(unicode charcter - .&quot;\uFFFD&quot;, replacement character)</p>
  <p>To do this we are submitting hive jobs to cluster.
  Tried with Lazy simple serde using ROW FORMAT Delimited -</p>
  <blockquote>
  <p>gcloud dataproc jobs submit hive --cluster  --region
   --execute &quot;CREATE EXTERNAL TABLE hiveuni_test_01(codes
  string,telephone_num string,finding_name string,given_name
  string,alt_finding_name string,house_num string,street_name
  string,locality string,state string,reserved string,zip_code
  string,directive_text string,special_listing_text string,id
  string,latitude string,longitude string,rboc_sent_date string)  ROW
  FORMAT DELIMITED FIELDS TERMINATED BY '\uFFFD' LINES TERMINATED BY
  '\n' STORED AS TEXTFILE LOCATION
  'gs://hive-idaas-dev-warehouse/datasets/unicode_file';&quot;</p>
  </blockquote>
  <p>But this does not create the table correctly , entire row is put to the first column only.</p>
  <p>We are using cloud SQL mysql server as hive metastore , checked that mysql has utf8 encoding also.</p>
  <p>Tried with multidelimitserde -</p>
  <blockquote>
  <p>gcloud dataproc jobs submit hive --cluster
  dev-sm-35cb3516-ed82-4ec2-bf0d-89bd7e0e60f0 --region us-central1
  --jars gs://hive-idaas-dev-warehouse/hive-jar/hive-contrib-0.14.0.jar --execute &quot;CREATE EXTERNAL TABLE hiveuni_test_05 (codes string,telephone_num string,finding_name string,given_name
  string,alt_finding_name string,house_num string,street_name
  string,locality string,state string,reserved string,zip_code
  string,directive_text string,special_listing_text string,id
  string,latitude string,longitude string,rboc_sent_date string)  ROW
  FORMAT SERDE 'org.apache.hadoop.hive.serde2.MultiDelimitSerDe'  WITH
  SERDEPROPERTIES ('field.delim'='\uFFFD') STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'  LOCATION
  'gs://hive-idaas-dev-warehouse/datasets/unicode_file';&quot;</p>
  </blockquote>
  <p>This gives an exception - <code>java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.serde2.MultiDelimitSerDe not found</code></p>
  <p>I have put an initialization script during start of the cluster which will place the <code>hive-contrib-0.14.0.jar</code> containing the class <code>org.apache.hadoop.hive.serde2.MultiDelimitSerDe</code> in <code>/usr/lib/hadoop/lib/</code>. I see that jar is placed in the folder by doing ssh to the cluster.</p>
  <p>Is there a way to read unicode characters by hive client while creating table or why do I still get an error classNotFound even after placing the jar in hadoop lib directory?</p>