<p>I created a startup probe and made it so it would <em>always</em> fail. It should result in the pod getting killed and restarting, but it doesn't. I see one event of the startup probe failing <em>(and no events after that)</em>, but the pods show as <code>1/1 Running</code>. And when I run my Helm test, it passes!</p>
<p>I guaranteed the failure by setting an invalid username <em>and</em> password for the startup probe check.</p>
<p><strong>Using K8s version: 1.19.4</strong></p>
<p>When I check events, I get:</p>
<pre><code>4m44s       Normal    SuccessfulCreate    replicaset/mysqlpod-5957645967   Created pod: mysqlpod-5957645967-fj95t
4m44s       Normal    ScalingReplicaSet   deployment/mysqlpod              Scaled up replica set mysqlpod-5957645967 to 1
4m44s       Normal    Scheduled           pod/mysqlpod-5957645967-fj95t    Successfully assigned data-layer/mysqlpod-5957645967-fj95t to minikube
4m43s       Normal    Created             pod/mysqlpod-5957645967-fj95t    Created container mysql
4m43s       Normal    Pulled              pod/mysqlpod-5957645967-fj95t    Container image &quot;mysql:5.6&quot; already present on machine
4m43s       Normal    Started             pod/mysqlpod-5957645967-fj95t    Started container mysql
4m41s       Warning   Unhealthy           pod/mysqlpod-5957645967-fj95t    Startup probe failed: Warning: Using a password on the command line interface can be insecure.
mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
</code></pre>
<p>Checking the Pods, I see (using <code>--watch</code>):</p>
<pre><code>NAME                            READY   STATUS    RESTARTS   AGE
mysql-db-app-5957645967-fj95t   0/1     Running   0          7m18s
mysql-db-app-5957645967-fj95t   1/1     Running   0          7m43s
</code></pre>
<p>Notice it has <strong>zero</strong> restarts.</p>
<p>My Deployment has:</p>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include &quot;mysqlapp.name&quot; . }}
  namespace: {{ quote .Values.metadata.namespace }}
spec:
  replicas: {{ .Values.deploymentSpecs.replicas}}
  selector:
    matchLabels:
      {{- include &quot;mysqlapp.selectorLabels&quot; . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include &quot;mysqlapp.selectorLabels&quot; . | nindent 8 }}
    spec:
      containers:
        - image: &quot;{{ .Values.image.name }}:{{ .Values.image.tag }}&quot;
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          name: {{ .Values.image.name }}
          env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom: 
                secretKeyRef:
                  name: db-credentials
                  key: db-password
          ports:
            - containerPort: {{ .Values.ports.containerPort }}
              name: {{ .Values.image.name }}
          startupProbe:
            exec:
              command:
                - /bin/sh
                - -c
                - mysqladmin ping -u wrong -pwrong
            periodSeconds: {{ .Values.startupProbe.periodSeconds }}
            timeoutSeconds: {{ .Values.startupProbe.timeoutSeconds }}
            successThreshold: {{ .Values.startupProbe.successThreshold }}
            failureThreshold: {{ .Values.startupProbe.failureThreshold }}
</code></pre>
<p>Notice the <code>- mysqladmin ping -u wrong -pwrong</code> above.</p>
<p>Values.yaml:</p>
<pre class="lang-yaml prettyprint-override"><code>metadata:
  namespace: data-layer
  myprop: value
deploymentSpecs:
  replicas: 1
labels:
  app: db-service
image:
  name: mysql
  pullPolicy: IfNotPresent
  tag: &quot;5.6&quot;
ports:
  containerPort: 3306
startupProbe:
  periodSeconds: 10
  timeoutSeconds: 2
  successThreshold: 1
  failureThreshold: 5
</code></pre>
<p>Even waiting for 5 minutes, I'm still able to run the test (which uses a MySql client to reach the DB) and it works! Why won't this fail?</p>

<p>It wasn't failing because it turns out the <code>ping</code> command returns a <code>0</code> status even if the user/pass is wrong, as long as it can reach the server.</p>
<p><a href="https://dev.mysql.com/doc/refman/8.0/en/mysqladmin.html" rel="nofollow noreferrer">MySql ping command</a></p>
<blockquote>
<p>Check whether the server is available. The return status from mysqladmin is 0 if the server is running, 1 if it is not. This is 0 even in case of an error such as Access denied, because this means that the server is running but refused the connection, which is different from the server not running.</p>
</blockquote>
<p>To force a failure and restarts, you could use:</p>
<pre class="lang-sh prettyprint-override"><code>mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD} --host fake
</code></pre>
