<p>I try to use SparkKubernetesOperator to run spark job into Kubernetes with the same DAG and yaml files as the following question:</p>
<p><a href="https://stackoverflow.com/questions/68371840/unable-to-create-sparkapplications-on-kubernetes-cluster-using-sparkkubernetesop/69129609#69129609">Unable to create SparkApplications on Kubernetes cluster using SparkKubernetesOperator from Airflow DAG</a></p>
<p>But airflow shows the following error:</p>
<pre><code>HTTP response headers: HTTPHeaderDict({'Audit-Id': 'e2e1833d-a1a6-40d4-9d05-104a32897deb', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Fri, 10 Sep 2021 08:38:33 GMT', 'Content-Length': '462'})
HTTP response body: {&quot;kind&quot;:&quot;Status&quot;,&quot;apiVersion&quot;:&quot;v1&quot;,&quot;metadata&quot;:{},&quot;status&quot;:&quot;Failure&quot;,&quot;message&quot;:&quot;the object provided is unrecognized (must be of type SparkApplication): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string \&quot;json:\\\&quot;apiVersion,omitempty\\\&quot;\&quot;; Kind string \&quot;json:\\\&quot;kind,omitempty\\\&quot;\&quot; } (222f7573722f6c6f63616c2f616972666c6f772f646167732f636f6e6669 ...)&quot;,&quot;reason&quot;:&quot;BadRequest&quot;,&quot;code&quot;:400}
</code></pre>
<p>Any suggestion to resolve that problem???</p>

Carlos Liarte
<p>think u had the same problem like me</p>
<pre><code>SparkKubernetesOperator(
  task_id='spark_pi_submit',
  namespace=&quot;default&quot;,
  application_file=open(&quot;/opt/airflow/dags/repo/script/spark-test.yaml&quot;).read(), #officially know bug
  kubernetes_conn_id=&quot;kubeConnTest&quot;, #ns default in airflow connection UI
  do_xcom_push=True,
  dag=dag
)
</code></pre>
<p>I wrapped it like this.
and it works like charm</p>
<p><a href="https://github.com/apache/airflow/issues/17371" rel="nofollow noreferrer">https://github.com/apache/airflow/issues/17371</a></p>
