<p>I'm trying to install VerneMQ on a Kubernetes cluster over Oracle OCI usign Helm chart.</p>

<p>The Kubernetes infrastructure seems to be up and running, I can deploy my custom microservices without a problem.</p>

<p>I'm following the instructions from <a href="https://github.com/vernemq/docker-vernemq" rel="nofollow noreferrer">https://github.com/vernemq/docker-vernemq</a></p>

<p>Here the steps:</p>

<ul>
<li><code>helm install --name="broker" ./</code> from helm/vernemq directory</li>
</ul>

<p>the output is:</p>

<pre><code>NAME:   broker
LAST DEPLOYED: Fri Mar  1 11:07:37 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==&gt; v1/RoleBinding
NAME            AGE
broker-vernemq  1s

==&gt; v1/Service
NAME                     TYPE       CLUSTER-IP    EXTERNAL-IP  PORT(S)   AGE
broker-vernemq-headless  ClusterIP  None          &lt;none&gt;       4369/TCP  1s
broker-vernemq           ClusterIP  10.96.120.32  &lt;none&gt;       1883/TCP  1s

==&gt; v1/StatefulSet
NAME            DESIRED  CURRENT  AGE
broker-vernemq  3        1        1s

==&gt; v1/Pod(related)
NAME              READY  STATUS             RESTARTS  AGE
broker-vernemq-0  0/1    ContainerCreating  0         1s

==&gt; v1/ServiceAccount
NAME            SECRETS  AGE
broker-vernemq  1        1s

==&gt; v1/Role
NAME            AGE
broker-vernemq  1s


NOTES:
1. Check your VerneMQ cluster status:
  kubectl exec --namespace default broker-vernemq-0 /usr/sbin/vmq-admin cluster show

2. Get VerneMQ MQTT port
  echo "Subscribe/publish MQTT messages there: 127.0.0.1:1883"
  kubectl port-forward svc/broker-vernemq 1883:1883
</code></pre>

<p>but when I do this check</p>

<p><code>kubectl exec --namespace default broker-vernemq-0 vmq-admin cluster show</code></p>

<p>I got</p>

<pre><code>Node 'VerneMQ@broker-vernemq-0..default.svc.cluster.local' not responding to pings.
command terminated with exit code 1
</code></pre>

<p>I think there is something wrong with subdomain (the double dots without nothing between them)</p>

<p>Whit this command</p>

<pre><code>kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name | head -1) -c kubedns
</code></pre>

<p>The last log line is</p>

<pre><code>I0301 10:07:38.366826       1 dns.go:552] Could not find endpoints for service "broker-vernemq-headless" in namespace "default". DNS records will be created once endpoints show up.
</code></pre>

<p>I've also tried with this  custom yaml:</p>

<pre><code>apiVersion: apps/v1
kind: StatefulSet
metadata:
  namespace: default
  name: vernemq
  labels:
    app: vernemq
spec:
  serviceName: vernemq
  replicas: 3
  selector:
    matchLabels:
      app: vernemq
  template:
    metadata:
      labels:
        app: vernemq
    spec:
      containers:
      - name: vernemq
        image: erlio/docker-vernemq:latest
        imagePullPolicy: Always
        ports:
          - containerPort: 1883
            name: mqtt
          - containerPort: 8883
            name: mqtts
          - containerPort: 4369
            name: epmd
        env:
        - name: DOCKER_VERNEMQ_KUBERNETES_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: DOCKER_VERNEMQ_ALLOW_ANONYMOUS
          value: "off"
        - name: DOCKER_VERNEMQ_DISCOVERY_KUBERNETES
          value: "1"
        - name: DOCKER_VERNEMQ_KUBERNETES_APP_LABEL
          value: "vernemq"
        - name: DOCKER_VERNEMQ_VMQ_PASSWD__PASSWORD_FILE
          value: "/etc/vernemq-passwd/vmq.passwd"
        volumeMounts:
          - name: vernemq-passwd
            mountPath: /etc/vernemq-passwd
            readOnly: true

      volumes:
      - name: vernemq-passwd
        secret:
          secretName: vernemq-passwd
---
apiVersion: v1
kind: Service
metadata:
  name: vernemq
  labels:
    app: vernemq
spec:
  clusterIP: None
  selector:
    app: vernemq
  ports:
  - port: 4369
    name: epmd
---
apiVersion: v1
kind: Service
metadata:
  name: mqtt
  labels:
    app: mqtt
spec:
  type: ClusterIP
  selector:
    app: vernemq
  ports:
  - port: 1883
    name: mqtt
---
apiVersion: v1
kind: Service
metadata:
  name: mqtts
  labels:
    app: mqtts
spec:
  type: LoadBalancer
  selector:
    app: vernemq
  ports:
  - port: 8883
    name: mqtts
</code></pre>

<p>Any suggestion?</p>

<p>Many thanks<br>
Jack</p>

Jack
<p>It seems to be a bug in the Docker image. The suggestion on github is to built your own image or use the later VerneMQ image (after 1.6.x) where it has been fixed.</p>

<p>Suggestion mentioned here: <a href="https://github.com/vernemq/docker-vernemq/pull/92" rel="nofollow noreferrer">https://github.com/vernemq/docker-vernemq/pull/92</a></p>

<p>Pull-Request for a possible fix: <a href="https://github.com/vernemq/docker-vernemq/pull/97" rel="nofollow noreferrer">https://github.com/vernemq/docker-vernemq/pull/97</a></p>

<p>EDIT: </p>

<p>I only got it to work without helm. Using <code>kubectl create -f ./cluster.yaml</code>, with the following <code>cluster.yaml</code>:</p>

<pre><code>---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: vernemq
  namespace: default
spec:
  serviceName: vernemq
  replicas: 3
  selector:
    matchLabels:
      app: vernemq
  template:
    metadata:
      labels:
        app: vernemq
    spec:
      serviceAccountName: vernemq
      containers:
      - name: vernemq
        image: erlio/docker-vernemq:latest
        ports:
        - containerPort: 1883
          name: mqttlb
        - containerPort: 1883
          name: mqtt
        - containerPort: 4369
          name: epmd
        - containerPort: 44053
          name: vmq
        - containerPort: 9100
        - containerPort: 9101
        - containerPort: 9102
        - containerPort: 9103
        - containerPort: 9104
        - containerPort: 9105
        - containerPort: 9106
        - containerPort: 9107
        - containerPort: 9108
        - containerPort: 9109
        env:
        - name: DOCKER_VERNEMQ_DISCOVERY_KUBERNETES
          value: "1"
        - name: DOCKER_VERNEMQ_KUBERNETES_APP_LABEL
          value: "vernemq"
        - name: DOCKER_VERNEMQ_KUBERNETES_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MY_POD_NAME
          valueFrom:
           fieldRef:
             fieldPath: metadata.name
        - name: DOCKER_VERNEMQ_ERLANG__DISTRIBUTION__PORT_RANGE__MINIMUM
          value: "9100"
        - name: DOCKER_VERNEMQ_ERLANG__DISTRIBUTION__PORT_RANGE__MAXIMUM
          value: "9109"
        - name: DOCKER_VERNEMQ_KUBERNETES_INSECURE
          value: "1"
        # only allow anonymous access for development / testing purposes!
        # - name: DOCKER_VERNEMQ_ALLOW_ANONYMOUS
        #   value: "on"
---
apiVersion: v1
kind: Service
metadata:
  name: vernemq
  labels:
    app: vernemq
spec:
  clusterIP: None
  selector:
    app: vernemq
  ports:
  - port: 4369
    name: empd
  - port: 44053
    name: vmq
---
apiVersion: v1
kind: Service
metadata:
  name: mqttlb
  labels:
    app: mqttlb
spec:
  type: LoadBalancer
  selector:
    app: vernemq
  ports:
  - port: 1883
    name: mqttlb
---
apiVersion: v1
kind: Service
metadata:
  name: mqtt
  labels:
    app: mqtt
spec:
  type: NodePort
  selector:
    app: vernemq
  ports:
  - port: 1883
    name: mqtt
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: vernemq
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: endpoint-reader
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["endpoints", "deployments", "replicasets", "pods"]
  verbs: ["get", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: endpoint-reader
subjects:
- kind: ServiceAccount
  name: vernemq
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: endpoint-reader
</code></pre>

<p>Needs a few seconds to get the pods ready.</p>
