title: How to use Apache ActiveMQ Artemis in Kubernetes
body: <p>I have an issue where I have a workload in Kubernetes which contains an Apache ActiveMQ Artemis broker. The server starts properly when I have a single pod in my workload, the issue starts when I try to scale them. The brokers in the pods can't connect to each other, so I can't scale my work load. My final goal is to make it scalable. I tried it locally with two docker containers and it worked fine.</p>
  
  <p>Here is my <code>broker.xml</code>:</p>
  
  <pre class="lang-xml prettyprint-override"><code>&lt;configuration xmlns="urn:activemq"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:xi="http://www.w3.org/2001/XInclude"
                 xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd"&gt;
  
      &lt;core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="urn:activemq:core "&gt;
  
          &lt;name&gt;Broker1&lt;/name&gt;
  
          &lt;broadcast-groups&gt;
              &lt;broadcast-group name="brokerCluster-broadcast"&gt;
                  &lt;local-bind-address&gt;0.0.0.0&lt;/local-bind-address&gt;
                  &lt;local-bind-port&gt;10000&lt;/local-bind-port&gt;
                  &lt;group-address&gt;231.7.7.7&lt;/group-address&gt;
                  &lt;group-port&gt;9876&lt;/group-port&gt;
                  &lt;broadcast-period&gt;20&lt;/broadcast-period&gt;
                  &lt;connector-ref&gt;netty-connector&lt;/connector-ref&gt;
              &lt;/broadcast-group&gt;
          &lt;/broadcast-groups&gt;
  
          &lt;discovery-groups&gt;
              &lt;discovery-group name="brokerCluster-discovery"&gt;
                  &lt;local-bind-port&gt;10000&lt;/local-bind-port&gt;
                  &lt;local-bind-address&gt;0.0.0.0&lt;/local-bind-address&gt;
                  &lt;group-address&gt;231.7.7.7&lt;/group-address&gt;
                  &lt;group-port&gt;9876&lt;/group-port&gt;
                  &lt;refresh-timeout&gt;10&lt;/refresh-timeout&gt;
              &lt;/discovery-group&gt;
          &lt;/discovery-groups&gt;
  
          &lt;cluster-connections&gt;
              &lt;cluster-connection name="brokerCluster"&gt;
                  &lt;connector-ref&gt;netty-connector&lt;/connector-ref&gt;
                  &lt;retry-interval&gt;500&lt;/retry-interval&gt;
                  &lt;use-duplicate-detection&gt;true&lt;/use-duplicate-detection&gt;
                  &lt;message-load-balancing&gt;ON_DEMAND&lt;/message-load-balancing&gt;
                  &lt;max-hops&gt;1&lt;/max-hops&gt;
                  &lt;discovery-group-ref discovery-group-name="brokerCluster-discovery"/&gt;
              &lt;/cluster-connection&gt;
          &lt;/cluster-connections&gt;
  
          &lt;connectors&gt;
              &lt;connector name="netty-connector"&gt;tcp://0.0.0.0:61610&lt;/connector&gt;
          &lt;/connectors&gt;
  
          &lt;persistence-enabled&gt;true&lt;/persistence-enabled&gt;
  
          &lt;journal-type&gt;NIO&lt;/journal-type&gt;
  
          &lt;paging-directory&gt;data/paging&lt;/paging-directory&gt;
  
          &lt;bindings-directory&gt;data/bindings&lt;/bindings-directory&gt;
  
          &lt;journal-directory&gt;data/journal&lt;/journal-directory&gt;
  
          &lt;large-messages-directory&gt;data/large-messages&lt;/large-messages-directory&gt;
  
          &lt;journal-datasync&gt;true&lt;/journal-datasync&gt;
  
          &lt;journal-min-files&gt;2&lt;/journal-min-files&gt;
  
          &lt;journal-pool-files&gt;10&lt;/journal-pool-files&gt;
  
          &lt;journal-device-block-size&gt;4096&lt;/journal-device-block-size&gt;
  
          &lt;journal-file-size&gt;10M&lt;/journal-file-size&gt;
  
          &lt;journal-buffer-timeout&gt;536000&lt;/journal-buffer-timeout&gt;
  
          &lt;disk-scan-period&gt;5000&lt;/disk-scan-period&gt;
  
          &lt;max-disk-usage&gt;90&lt;/max-disk-usage&gt;
  
          &lt;critical-analyzer&gt;true&lt;/critical-analyzer&gt;
  
          &lt;critical-analyzer-timeout&gt;120000&lt;/critical-analyzer-timeout&gt;
  
          &lt;critical-analyzer-check-period&gt;60000&lt;/critical-analyzer-check-period&gt;
  
          &lt;critical-analyzer-policy&gt;HALT&lt;/critical-analyzer-policy&gt;
  
  
          &lt;page-sync-timeout&gt;536000&lt;/page-sync-timeout&gt;
  
          &lt;acceptors&gt;
  
              &lt;acceptor name="netty-acceptor"&gt;tcp://0.0.0.0:61610&lt;/acceptor&gt;
  
              &lt;acceptor name="artemis"&gt;tcp://0.0.0.0:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true&lt;/acceptor&gt;
  
              &lt;acceptor name="amqp"&gt;tcp://0.0.0.0:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true&lt;/acceptor&gt;
  
              &lt;acceptor name="stomp"&gt;tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true&lt;/acceptor&gt;
  
              &lt;acceptor name="hornetq"&gt;tcp://0.0.0.0:5445?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;protocols=HORNETQ,STOMP;useEpoll=true&lt;/acceptor&gt;
  
              &lt;acceptor name="mqtt"&gt;tcp://0.0.0.0:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true&lt;/acceptor&gt;
  
          &lt;/acceptors&gt;
  
  
          &lt;security-settings&gt;
              &lt;security-setting match="#"&gt;
                  &lt;permission type="createNonDurableQueue" roles="amq"/&gt;
                  &lt;permission type="deleteNonDurableQueue" roles="amq"/&gt;
                  &lt;permission type="createDurableQueue" roles="amq"/&gt;
                  &lt;permission type="deleteDurableQueue" roles="amq"/&gt;
                  &lt;permission type="createAddress" roles="amq"/&gt;
                  &lt;permission type="deleteAddress" roles="amq"/&gt;
                  &lt;permission type="consume" roles="amq"/&gt;
                  &lt;permission type="browse" roles="amq"/&gt;
                  &lt;permission type="send" roles="amq"/&gt;
                  &lt;permission type="manage" roles="amq"/&gt;
              &lt;/security-setting&gt;
          &lt;/security-settings&gt;
  
          &lt;address-settings&gt;
              &lt;address-setting match="activemq.management#"&gt;
                  &lt;dead-letter-address&gt;DLQ&lt;/dead-letter-address&gt;
                  &lt;expiry-address&gt;ExpiryQueue&lt;/expiry-address&gt;
                  &lt;redelivery-delay&gt;0&lt;/redelivery-delay&gt;
                  &lt;max-size-bytes&gt;-1&lt;/max-size-bytes&gt;
                  &lt;message-counter-history-day-limit&gt;10&lt;/message-counter-history-day-limit&gt;
                  &lt;address-full-policy&gt;PAGE&lt;/address-full-policy&gt;
                  &lt;auto-create-queues&gt;true&lt;/auto-create-queues&gt;
                  &lt;auto-create-addresses&gt;true&lt;/auto-create-addresses&gt;
                  &lt;auto-create-jms-queues&gt;true&lt;/auto-create-jms-queues&gt;
                  &lt;auto-create-jms-topics&gt;true&lt;/auto-create-jms-topics&gt;
              &lt;/address-setting&gt;
              &lt;address-setting match="#"&gt;
                  &lt;dead-letter-address&gt;DLQ&lt;/dead-letter-address&gt;
                  &lt;expiry-address&gt;ExpiryQueue&lt;/expiry-address&gt;
                  &lt;redelivery-delay&gt;0&lt;/redelivery-delay&gt;
                  &lt;redistribution-delay&gt;0&lt;/redistribution-delay&gt;
                  &lt;max-size-bytes&gt;-1&lt;/max-size-bytes&gt;
                  &lt;message-counter-history-day-limit&gt;10&lt;/message-counter-history-day-limit&gt;
                  &lt;address-full-policy&gt;PAGE&lt;/address-full-policy&gt;
                  &lt;auto-create-queues&gt;true&lt;/auto-create-queues&gt;
                  &lt;auto-create-addresses&gt;true&lt;/auto-create-addresses&gt;
                  &lt;auto-create-jms-queues&gt;true&lt;/auto-create-jms-queues&gt;
                  &lt;auto-create-jms-topics&gt;true&lt;/auto-create-jms-topics&gt;
              &lt;/address-setting&gt;
          &lt;/address-settings&gt;
  
          &lt;addresses&gt;
              &lt;address name="DLQ"&gt;
                  &lt;anycast&gt;
                      &lt;queue name="DLQ" /&gt;
                  &lt;/anycast&gt;
              &lt;/address&gt;
              &lt;address name="ExpiryQueue"&gt;
                  &lt;anycast&gt;
                      &lt;queue name="ExpiryQueue" /&gt;
                  &lt;/anycast&gt;
              &lt;/address&gt;
              &lt;address name="TestQueue"&gt;
                  &lt;anycast&gt;
                      &lt;queue name="testQueue" /&gt;
                  &lt;/anycast&gt;
              &lt;/address&gt;
          &lt;/addresses&gt;
      &lt;/core&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>Edit: Attached kubernetes,docker configs</p>
  
  <p><strong>deployment.yml</strong></p>
  
  <pre><code>apiVersion: v1
  kind: Service
  metadata:
    name: artemis
    labels:
      app: artemis
  spec:
    ports:
    - port: 6161
      name: service
      protocol: UDP
    - port: 8161
      name: console
      protocol: UDP
    - port: 9876
      name: broadcast
      protocol: UDP
    - port: 61610
      name: netty-connector
      protocol: TCP
    - port: 5672
      name: acceptor-amqp
      protocol: TCP
    - port: 61613
      name: acceptor-stomp
      protocol: TCP
    - port: 5445
      name: accep-hornetq
      protocol: TCP
    - port: 1883
      name: acceptor-mqt
      protocol: TCP
    - port: 10000
      protocol: UDP
      name: brokercluster-broadcast // this name is invalid but i wanted to match it to my broker.xml
    clusterIP: None
    selector:
      app: artemis01
  ---
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: artemis01headless
    namespace: artemis
  spec:
    selector:
      matchLabels:
        app: artemis01 
    serviceName: artemis01
    replicas: 3
    template:
      metadata:
        labels:
          app: artemis01 
      spec:
        affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                - matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - worker
        containers:
          - env:
            - name: ARTEMIS_PASSWORD
              value: admin
            - name: ARTEMIS_USER
              value: admin
            - name: MY_POD_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP
            name: artemis
            image: 
            ports:
            - containerPort: 6161
              name: service
              protocol: UDP
            - containerPort: 8161
              name: console
              protocol: UDP
            - containerPort: 9876
              name: broadcast
              protocol: UDP
            - containerPort: 61610
              name: netty-connector
              protocol: TCP
            - containerPort: 5672
              name: acceptor-amqp
              protocol: TCP
            - containerPort: 61613
              name: acceptor-stomp
              protocol: TCP
            - containerPort: 5445
              name: accep-hornetq
              protocol: TCP
            - containerPort: 1883
              name: acceptor-mqtt
              protocol: TCP
            - containerPort: 10000
              name: brokercluster-broadcast
              protocol: UDP
        imagePullSecrets:
          - name: xxxxxxx
  
  </code></pre>
  
  <p><a href="https://github.com/apache/activemq-artemis/tree/master/artemis-docker" rel="nofollow noreferrer"><strong>Dockerfile</strong> source</a> </p>
  
  <pre><code># Licensed to the Apache Software Foundation (ASF) under one
  # or more contributor license agreements.  See the NOTICE file
  # distributed with this work for additional information
  # regarding copyright ownership.  The ASF licenses this file
  # to you under the Apache License, Version 2.0 (the
  # "License"); you may not use this file except in compliance
  # with the License.  You may obtain a copy of the License at
  #
  #   http://www.apache.org/licenses/LICENSE-2.0
  #
  # Unless required by applicable law or agreed to in writing,
  # software distributed under the License is distributed on an
  # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  # KIND, either express or implied.  See the License for the
  # specific language governing permissions and limitations
  # under the License.
  
  # ActiveMQ Artemis
  
  FROM jboss/base-jdk:8
  LABEL maintainer="Apache ActiveMQ Team"
  # Make sure pipes are considered to determine success, see: https://github.com/hadolint/hadolint/wiki/DL4006
  SHELL ["/bin/bash", "-o", "pipefail", "-c"]
  WORKDIR /opt
  
  ENV ARTEMIS_USER artemis
  ENV ARTEMIS_PASSWORD artemis
  ENV ANONYMOUS_LOGIN false
  ENV CREATE_ARGUMENTS --user ${ARTEMIS_USER} --password ${ARTEMIS_PASSWORD} --silent --http-host 0.0.0.0 --relax-jolokia
  
  USER root
  
  # add user and group for artemis
  RUN groupadd -g 1001 -r artemis &amp;&amp; useradd -r -u 1001 -g artemis artemis \
   &amp;&amp; yum install -y libaio &amp;&amp; yum -y clean all
  
  USER artemis
  
  ADD . /opt/activemq-artemis
  
  # Web Server
  EXPOSE 8161 \
      61610 \
      9876 \
      61613 \
      61616 \
      5672 \
      5445 \
      1883 \
      10000
  
  USER root
  
  RUN mkdir /var/lib/artemis-instance &amp;&amp; chown -R artemis.artemis /var/lib/artemis-instance
  
  COPY ./docker/docker-run.sh /
  
  USER artemis
  
  # Expose some outstanding folders
  VOLUME ["/var/lib/artemis-instance"]
  WORKDIR /var/lib/artemis-instance
  
  ENTRYPOINT ["/docker-run.sh"]
  CMD ["run"]
  
  </code></pre>
  
  <p><strong>run.sh</strong></p>
  
  <pre class="lang-sh prettyprint-override"><code>set -e
  
  BROKER_HOME=/var/lib/
  CONFIG_PATH=$BROKER_HOME/etc
  export BROKER_HOME OVERRIDE_PATH CONFIG_PATH
  
  echo CREATE_ARGUMENTS=${CREATE_ARGUMENTS}
  
  if ! [ -f ./etc/broker.xml ]; then
      /opt/activemq-artemis/bin/artemis create ${CREATE_ARGUMENTS} .
      #the script copies my broker.xml to /var/lib/artemis-instance/etc/broker.xml here.
      sed -i -e 's|$PLACEHOLDERIP|'$MY_POD_IP'|g' /var/lib/artemis-instance/etc/broker.xml
  else
      echo "broker already created, ignoring creation"
  fi
  
  exec ./bin/artemis "$@"
  </code></pre>