<p>Is it actually possible to use a kubernates secret in a terraform-deployed application. I am seeing some odd behaviour.</p>

<p>I define a cluster with appropriate node pool, a config map and a secret. The secret contains the service account key json data.
I can then deploy my application using <code>kubectl apply -f myapp-deploy.yaml</code> and it works fine. That tells me the cluster is all good, including the secret and config.
However, when I try to deploy with terraform I get an error in what looks like the service account fetch:</p>

<pre><code>2019-07-19 06:20:45.497  INFO [myapp,,,] 1 --- [main] b.c.PropertySourceBootstrapConfiguration : Located property source: SecretsPropertySource {name='secrets.myapp.null'} 
2019-07-19 06:20:45.665  WARN [myapp,,,] 1 --- [main] io.fabric8.kubernetes.client.Config      : Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
2019-07-19 06:20:45.677  INFO [myapp,,,] 1 --- [main] n.c.m.s.myappApplication          : The following profiles are active: test-dev
</code></pre>

<p>The middle line is the interesting one, it seems to be trying to read the service account from the wrong place.</p>

<p>I've walked the relevant settings from my yaml file over to my tf file but maybe I missed something. Here's what the yaml file looks like:</p>

<pre><code>...
          env:
          - name: GOOGLE_APPLICATION_CREDENTIALS
            value: "/var/run/secret/cloud.google.com/myapp-sa.json"
          volumeMounts:
            - name: "service-account"
              mountPath: "/var/run/secret/cloud.google.com"
          ports:
            - containerPort: 8080
      volumes:
        - name: "service-account"
          secret:
            secretName: "myapp"
...
</code></pre>

<p>And this yaml basically just works fine.
Now the equivalent in my tf file looks like:</p>

<pre><code>...
          env {
            name  = "GOOGLE_APPLICATION_CREDENTIALS"
            value = "/var/run/secret/cloud.google.com/myapp-sa.json"
          }
          volume_mount {
            name       = "myapp-sa"
            mount_path = "/var/run/secret/cloud.google.com"
            sub_path = ""
          }
        }
        volume {
          name = "myapp-sa"
          secret {
            secret_name = "myapp"
          }
        }
...
</code></pre>

<p>And this gives the above error. It seems to decide to look in <code>/var/run/secrets/kubernetes.io/serviceaccount/token</code> for the service account token instead of where I told it to. But only when deployed by <strong>terraform</strong>. I'm deploying the same image, and into the same cluster with the same <strong>configmap</strong>. There's something wrong with my tf somewhere. I've tried importing from the yaml deploy but I couldn't see anything important that I missed.</p>

<p>FWIW this is a Spring Boot application running on GKE.</p>

<p>Hopefully someone knows the answer.</p>

<h2>Thanks for any help.</h2>

<p>more info: I turned on debugging for io.fabric8.kubernetes and reran both scenarios ie <strong>terraform</strong> and yaml file. Here are the relevant log snippets:</p>

<p>Terraform:</p>

<pre><code>2019-07-23 23:03:39.189 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Trying to configure client from Kubernetes config...
2019-07-23 23:03:39.268 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Did not find Kubernetes config at: [/root/.kube/config]. Ignoring.
2019-07-23 23:03:39.274 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Trying to configure client from service account...
2019-07-23 23:03:39.274 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Found service account host and port: 10.44.0.1:443
2019-07-23 23:03:39.282 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Did not find service account ca cert at: [/var/run/secrets/kubernetes.io/serviceaccount/ca.crt].
2019-07-23 23:03:39.285  WARN [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Error reading service account token from: [/var/run/secrets/kubernetes.io/serviceaccount/token]. Ignoring.
2019-07-23 23:03:39.291 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Trying to configure client namespace from Kubernetes service account namespace path...
2019-07-23 23:03:39.295 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Did not find service account namespace at: [/var/run/secrets/kubernetes.io/serviceaccount/namespace]. Ignoring.
</code></pre>

<p>Yaml:</p>

<pre><code>2019-07-23 23:14:53.374 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Trying to configure client namespace from Kubernetes service account namespace path...
2019-07-23 23:14:53.375 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Found service account namespace at: [/var/run/secrets/kubernetes.io/serviceaccount/namespace].
2019-07-23 23:14:53.376 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Trying to configure client from Kubernetes config...
2019-07-23 23:14:53.377 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Did not find Kubernetes config at: [/root/.kube/config]. Ignoring.
2019-07-23 23:14:53.378 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Trying to configure client from service account...
2019-07-23 23:14:53.378 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Found service account host and port: 10.44.0.1:443
2019-07-23 23:14:53.383 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Found service account ca cert at: [/var/run/secrets/kubernetes.io/serviceaccount/ca.crt].
2019-07-23 23:14:53.384 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Found service account token at: [/var/run/secrets/kubernetes.io/serviceaccount/token].
2019-07-23 23:14:53.384 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Trying to configure client namespace from Kubernetes service account namespace path...
2019-07-23 23:14:53.384 DEBUG [snakecharmer,,,] 1 --- [           main] io.fabric8.kubernetes.client.Config      : Found service account namespace at: [/var/run/secrets/kubernetes.io/serviceaccount/namespace].
</code></pre>

<p>It looks like the yaml deploy finds what it needs at <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> etc and the <strong>terraform</strong> deploy doesn't. As if there is a phantom volume mount in there that is missing in ** terraform**</p>

<p>I found the fix. The <strong>terraform</strong> deploy adds a <code>automount_service_account_token = false</code> but the yaml default is for <code>true</code> and that makes all the difference.</p>

<p>The switch is in the template.spec section of the <code>kubernetes_deployment</code> in my tf file and that now looks like this snippet:</p>

<pre><code>...
      spec {
        restart_policy = "Always"
        automount_service_account_token = true
        container {
          port {
            container_port = 8080
            protocol       = "TCP"
          }
...
</code></pre>

<p>Setting the <code>automount_service_account_token = true</code> is the fix and it comes up fine with that in place.</p>
