<p>I want to share some secrets with my Kubeflow pipeline so I can use them as environment variables in my containers. I've written a pipeline-secrets.yaml that looks like this:</p>
<pre><code>apiVersion: v1
kind: Secret
metadata:
  name: pipeline-secrets
  namespace: kubeflow
type: Opaque
data:
  mysql_db_name: &lt;SECRET&gt;
  mysql_username: &lt;SECRET&gt;
  mysql_password: &lt;SECRET&gt;
  mysql_endpoint: &lt;SECRET&gt;
</code></pre>
<p>and a pipeline-pod-defaults.yaml that looks like this:</p>
<pre><code>apiVersion: kubeflow.org/v1alpha1
kind: PodDefault
metadata:
  name: pipeline-pod-defaults
  namespace: kubeflow
specs:
  desc: Configure pipeline secrets as environment variables
  env:
  - name: MYSQL_DB_NAME
    valueFrom:
      secretKeyRef:
        name: pepeline-secrets
        key: mysql_db_name
  - name: MYSQL_USER_NAME
    valueFrom:
      secretKeyRef:
        name: pepeline-secrets
        key: mysql_username
  - name: MYSQL_PASSWORD
    valueFrom:
      secretKeyRef:
        name: pepeline-secrets
        key: mysql_password
  - name: MYSQL_ENDPOINT
    valueFrom:
      secretKeyRef:
        name: pepeline-secrets
        key: mysql_endpoint
</code></pre>
<p>And this is how my pipeline looks like:</p>
<pre><code>import kfp
from kfp.dsl import ContainerOp
from kubernetes import client as k8s_client

@kfp.dsl.pipeline(
    name=&quot;Training pipeline&quot;,
    description=&quot;&quot;
)
def train_pipeline():
    get_data = ContainerOp(
        name=&quot;Get data&quot;,
        image=BASE_IMAGE,
        file_outputs={
            'data': 'data.csv'
        }
    )
    
    kfp.dsl.get_pipeline_conf().set_image_pull_secrets([
        k8s_client.V1ObjectReference(name=&quot;regcred&quot;),
        k8s_client.V1ObjectReference(name=&quot;pipeline-secrets&quot;),
    ])
    kfp.dsl.ResourceOp(
        name=&quot;pipeline-pod-defaults&quot;,
        k8s_resource=k8s_client.V1ObjectReference(name=&quot;pipeline-pod-defaults&quot;),
        action=&quot;apply&quot;
    )
</code></pre>
<p>But at the end I'm getting this error:</p>
<pre><code>This step is in Failed state with this message: error: error validating &quot;/tmp/manifest.yaml&quot;: error validating data: [apiVersion not set, kind not set]; if you choose to ignore these errors, turn validation off with --validate=false
</code></pre>
<p>Is this the correct approach? How can I share my secrets with the rest of the pipeline? Sorry if it's a newbie question, I'm new to both Kubernetes and Kubeflow</p>

João Areias
<p>So, in the end, what I did was write a get-data-components.yaml to create my component and wrote the function below and it worked:</p>
<pre><code>def build_get_data():
    component = kfp.components.load_component_from_file(os.path.join(COMPONENTS_PATH, 'get-data-component.yaml'))()
    component.add_volume(k8s_client.V1Volume(
        name=&quot;get-data-volume&quot;,
        secret=k8s_client.V1SecretVolumeSource(secret_name=&quot;pipeline-secrets&quot;))
    )
    envs = [
        (&quot;MYSQL_DB_NAME&quot;, &quot;mysql_db_name&quot;),
        (&quot;MYSQL_USER_NAME&quot;, &quot;mysql_username&quot;), 
        (&quot;MYSQL_PASSWORD&quot;, &quot;mysql_password&quot;), 
        (&quot;MYSQL_ENDPOINT&quot;, &quot;mysql_endpoint&quot;)
    ]
    for name, key in envs:
        component.add_env_variable(
            V1EnvVar(
                name=name,
                value_from=k8s_client.V1EnvVarSource(secret_key_ref=k8s_client.V1SecretKeySelector(
                    name=&quot;pipeline-secrets&quot;,
                    key=key
                    )
                )
            )
        )
    return component
</code></pre>
