title: Conflict between httpclient version and Apache Spark
body: <p>I'm developing a Java application using Apache Spark. I use this version:</p>
  
  <pre><code>&lt;dependency&gt;
      &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
      &lt;artifactId&gt;spark-core_2.10&lt;/artifactId&gt;
      &lt;version&gt;1.2.2&lt;/version&gt;
  &lt;/dependency&gt;
  </code></pre>
  
  <p>In my code, there is a transitional dependency:</p>
  
  <pre><code>&lt;dependency&gt;
      &lt;groupId&gt;org.apache.httpcomponents&lt;/groupId&gt;
      &lt;artifactId&gt;httpclient&lt;/artifactId&gt;
      &lt;version&gt;4.5.2&lt;/version&gt;
  &lt;/dependency&gt;
  </code></pre>
  
  <p>I package my application into a single JAR file. When deploying it on EC2 instance using <code>spark-submit</code>, I get this error.</p>
  
  <pre><code>Caused by: java.lang.NoSuchFieldError: INSTANCE
      at org.apache.http.conn.ssl.SSLConnectionSocketFactory.&lt;clinit&gt;(SSLConnectionSocketFactory.java:144)
      at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:87)
      at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
      at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
      at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:50)
      at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:38)
  </code></pre>
  
  <p>This error shows clearly that <code>SparkSubmit</code> has loaded an older version of the same Apache httpclient library and this conflict happens for this reason.</p>
  
  <p>What is a good way to solve this issue?</p>
  
  <p>For some reason, I cannot upgrade Spark on my Java code. However, I could do that with the EC2 cluster easily. Is it possible to deploy my java code on a cluster with a higher version say 1.6.1 version?</p>