<p>I have basic and fundamental question regards to Hashicorp Vault. I would like to inject some secrets (password to database) from Vault to container with Spring application.</p>
<p>I have prepared a specific annotations for using Vault with Kubernetes and everything works fine, saving PASSWORD variable as environment allows me using in application.properties file.</p>
<pre><code>template:
metadata:
  annotations:
    vault.hashicorp.com/agent-inject: &quot;true&quot;
    vault.hashicorp.com/role: &quot;myapp-role&quot;
    vault.hashicorp.com/agent-inject-secret-foo: &quot;secret/creds&quot;
    vault.hashicorp.com/agent-inject-template-foo: |
      {{`{{- with secret &quot;secret/creds&quot; -}}
      PASSWORD={{ .Data.passcode }}
      {{- end }}`}}
  labels:
    app.kubernetes.io/name: {{ $appName }}
    app.kubernetes.io/instance: {{ .Release.Name }}
spec:
  containers:
    - name: {{ $appName }}
      image: &quot;{{ .Values.vvvv.image.repository }}:{{ .Values.vvvv.image.tag }}&quot;
      command: [&quot;/bin/bash&quot;, &quot;-c&quot;,&quot;while read line; do export $line; done &lt; /vault/secrets/foo; /usr/local/tomcat/bin/catalina.sh run&quot;]
      volumeMounts:
        - name: application-properties
          mountPath: /usr/local/tomcat/lib/application.properties
          subPath: application.properties
      ports:
        - name: http
          containerPort: 8080
          protocol: TCP     
</code></pre>
<p>the question is pretty simple, does it make sense? inject agent saves the file with PLAIN text password in the /vault path, so everybody can see this secret... another question, how can I rotate credentials for application? should I use specific controllers inside the spring application right?</p>

corey
<p>I think it definitely makes sense as the purpose is to avoid hard coding credentials in the spec.</p>
<blockquote>
<p>inject agent saves the file with PLAIN text password in the /vault
path, so everybody can see this secret.</p>
</blockquote>
<p>Even in bare metal servers or cloud instances, credentials are saved in plain text. In case of k8s, it is inside a container. In both scenarios, you would control who can access your instances or k8s pods. Only authorised personnel shall have the required access to pods in production cluster(s).</p>
<blockquote>
<p>how can I rotate credentials for application</p>
</blockquote>
<p>The vault agent injector runs a sidecar container along with your app container in the same pod. Its purpose is to look for any changes in the vault secret at regular intervals. If you do a <code>kubectl describe po &lt;pod-name&gt;</code> you'll find a sidecar container <code>vault-agent</code> running.</p>
<pre><code>kubectl get po app-example-deployment-7c4b45cf8-4fkr7
NAME                                     READY   STATUS    RESTARTS   AGE
app-example-deployment-7c4b45cf8-4fkr7   2/2     Running   0          166m
</code></pre>
<p><code>kubectl describe pod app-example-deployment-7c4b45cf8-4fkr7</code> :</p>
<pre><code>...
vault-agent:
    Container ID:  docker://b6f9df32ed903d684c972401f41e15a8f6b1bec62aa111bfd9c693159af1ff09
    Image:         vault:1.7.0
    Image ID:      docker-pullable://vault@sha256:635cf1c3f9b10fe03aad375f94cc61f63d74a189662165285a8bf1c189ea04b8
    Port:          &lt;none&gt;
    Host Port:     &lt;none&gt;
    Command:
      /bin/sh
      -ec
    Args:
      echo ${VAULT_CONFIG?} | base64 -d &gt; /home/vault/config.json &amp;&amp; vault agent -config=/home/vault/config.json
    State:          Running
      Started:      Tue, 13 Apr 2021 15:40:10 +0100
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     500m
      memory:  128Mi
    Requests:
      cpu:     250m
      memory:  64Mi
    Environment:
      VAULT_LOG_LEVEL:   info
      VAULT_LOG_FORMAT:  standard
...
</code></pre>
<p>After successfully fetching the secret from vault during deployment:</p>
<pre><code>kubectl exec -it app-example-deployment-7c4b45cf8-4fkr7 -c app -- cat /vault/secrets/db-creds
mongodb+srv://testUser:testPass@test-5xxxx.mongodb.net/testDb
</code></pre>
<p>If I change the kv secret in Vault with password set to, let's say &quot;testPass2&quot;, I need not do anything as the <code>vault-agent</code> sidecar container will update it for me automatically.</p>
<pre><code>kubectl exec -it app-example-deployment-7c4b45cf8-4fkr7 -c app -- cat /vault/secrets/db-creds
mongodb+srv://testUser:testPass2@test-5xxxx.mongodb.net/testDb
</code></pre>
<p>In the <code>vault-agent</code> sidecar container logs, you'll see something like.</p>
<pre><code>kubectl logs app-example-deployment-7c4b45cf8-4fkr7 -c vault-agent --follow
2021-04-13T14:40:10.426Z [INFO]  sink.file: creating file sink
2021-04-13T14:40:10.426Z [INFO]  sink.file: file sink configured: path=/home/vault/.vault-token mode=-rw-r-----
==&gt; Vault agent started! Log data will stream in below:

==&gt; Vault agent configuration:

                     Cgo: disabled
               Log Level: info
                 Version: Vault v1.7.0
             Version Sha: 4e222b85c40a810b74400ee3c54449479e32bb9f

2021-04-13T14:40:10.426Z [INFO]  template.server: starting template server
[INFO] (runner) creating new runner (dry: false, once: false)
2021-04-13T14:40:10.427Z [INFO]  auth.handler: starting auth handler
2021-04-13T14:40:10.427Z [INFO]  auth.handler: authenticating
2021-04-13T14:40:10.427Z [INFO]  sink.server: starting sink server
[INFO] (runner) creating watcher
2021-04-13T14:40:10.437Z [INFO]  auth.handler: authentication successful, sending token to sinks
2021-04-13T14:40:10.437Z [INFO]  auth.handler: starting renewal process
2021-04-13T14:40:10.437Z [INFO]  template.server: template server received new token
[INFO] (runner) stopping
[INFO] (runner) creating new runner (dry: false, once: false)
[INFO] (runner) creating watcher
[INFO] (runner) starting
2021-04-13T14:40:10.437Z [INFO]  sink.file: token written: path=/home/vault/.vault-token
2021-04-13T14:40:10.439Z [INFO]  auth.handler: renewed auth token
[INFO] (runner) rendered &quot;(dynamic)&quot; =&gt; &quot;/vault/secrets/db-creds&quot;
2021-04-13T15:23:43.315Z [INFO]  auth.handler: renewed auth token
[INFO] (runner) rendered &quot;(dynamic)&quot; =&gt; &quot;/vault/secrets/db-creds&quot;
2021-04-13T16:07:16.191Z [INFO]  auth.handler: renewed auth token
</code></pre>
