title: AWS Error Message: Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4
body: <p>I am facing the following error while writing to S3 bucket using pyspark.</p>
  <blockquote>
  <p>com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: A0B0C0000000DEF0, AWS Error Code: InvalidArgument, AWS Error Message: Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.,</p>
  </blockquote>
  <p>I have applied server-side encryption using AWS KMS service on the S3 bucket.
  I am using the following spark-submit command -</p>
  <pre><code>spark-submit --packages com.amazonaws:aws-java-sdk-pom:1.10.34,org.apache.hadoop:hadoop-aws:2.7.2 --jars sample-jar sample_pyspark.py 
  </code></pre>
  <p>This is the sample code I am working on -</p>
  <pre><code>spark_context = SparkContext()
  sql_context = SQLContext(spark_context) 
  spark = SparkSession.builder.appName('abc').getOrCreate()
  hadoopConf = spark_context._jsc.hadoopConfiguration()
  hadoopConf.set(&quot;fs.s3a.impl&quot;, &quot;org.apache.hadoop.fs.s3a.S3AFileSystem&quot;)
  #Have a spark dataframe 'source_data
  source_data.coalesce(1).write.mode('overwrite').parquet(&quot;s3a://sample-bucket&quot;)
  </code></pre>
  <p>Note: Tried to load the spark-dataframe into s3 bucket [without server-side encryption enabled] and it was successful</p>