title: unable to bring up spark 2.1.0 manually on HDP 2.5.3
body: <p>I was testing my spark code on spark 2.0.0 and I hit a bug SPARK-17463 and I wanted to use spark 2.1.0 since the bug is fixed in this version.</p>
  
  <p>However, I am unable to bring up spark-shell with yarn client mode for spark 2.1.0</p>
  
  <p>I need to get 2.1.0 working on a HDP 2.5.3 cluster.</p>
  
  <p>It throws an exception:</p>
  
  <pre><code>    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  17/02/15 14:28:46 WARN Utils: Service 'SparkUI' could not bind on port 4045. Attempting port 4046.
      java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig
        at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:181)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:168)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:151)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:156)
        at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:509)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
        at org.apache.spark.repl.Main$.createSparkSession(Main.scala:95)
        ... 47 elided
      Caused by: java.lang.ClassNotFoundException: com.sun.jersey.api.client.config.ClientConfig
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 61 more
  &lt;console&gt;:14: error: not found: value spark
             import spark.implicits._
                            ^
  &lt;console&gt;:14: error: not found: value spark
             import spark.sql
                            ^
  </code></pre>