title: ProtocolException on writing to s3 from spark
body: <p>I am trying to read from hdfs location and save into an AWS s3 Bucket as follows :</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>   spark.sql("select * from  sometbl")
            .coalesce(1)
            .write.mode(SaveMode.Overwrite)
            .format("csv")
            .save(s"s3a://mybucket/foldername")</code></pre>
  </div>
  </div>
  </p>
  <p>But i get this Exception   :</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>Caused by: org.apache.spark.SparkException: Task failed while writing rows.
          at org.apache.spark.sql.execution.datasources.FileFormatWriter$.org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask(FileFormatWriter.scala:257)
          at org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$write$1.apply(FileFormatWriter.scala:170)
          at org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$write$1.apply(FileFormatWriter.scala:169)
          at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
          at org.apache.spark.scheduler.Task.run(Task.scala:123)
          at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
          at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
          at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
  Caused by: com.amazonaws.SdkClientException: Unable to execute HTTP request: null
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1113)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1063)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
          at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
          at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4229)
          at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4176)
          at com.amazonaws.services.s3.AmazonS3Client.copyObject(AmazonS3Client.java:1852)
          at com.amazonaws.services.s3.transfer.internal.CopyCallable.copyInOneChunk(CopyCallable.java:146)
          at com.amazonaws.services.s3.transfer.internal.CopyCallable.call(CopyCallable.java:134)
          at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:133)
          at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:44)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          ... 3 more
  Caused by: com.amazonaws.thirdparty.apache.http.client.ClientProtocolException
          at com.amazonaws.thirdparty.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
          at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
          at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
          at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1235)
          at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1055)
          ... 17 more
  Caused by: com.amazonaws.thirdparty.apache.http.ProtocolException: Content-Length header already present
          at com.amazonaws.thirdparty.apache.http.protocol.RequestContent.process(RequestContent.java:96)
          at com.amazonaws.thirdparty.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
          at com.amazonaws.thirdparty.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:182)
          at com.amazonaws.thirdparty.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
          ... 22 more</code></pre>
  </div>
  </div>
  
  Is there a bug in s3 client ? these are versions used<br />
  &quot;com.amazonaws&quot; % &quot;aws-java-sdk&quot; % &quot;1.7.4&quot;,
  &quot;org.apache.hadoop&quot; % &quot;hadoop-aws&quot; % &quot;2.7.3&quot;</p>