<p>WSO2 EI-6.5 is setup in k8s environment. It worked fine without any issue. But due to some reason now it crashing with the below exception. <code>[EI-Core] ERROR - DefaultAddressPicker [LOCAL] [wso2.ei.domain] [3.5.4] integrator: Name or service not known</code></p>

<p>Full Log:</p>

<blockquote>
  <p>[2019-09-06 05:07:38,292] [EI-Core]  INFO - JMSListener JMS listener
  started [2019-09-06 05:07:38,293] [EI-Core]  INFO -
  PassThroughHttpListener Starting Pass-through HTTP Listener...
  [2019-09-06 05:07:38,305] [EI-Core]  INFO -
  PassThroughListeningIOReactorManager Pass-through HTTP Listener
  started on 0.0.0.0:8280 [2019-09-06 05:07:38,305] [EI-Core]  INFO -
  PassThroughHttpSSLListener Starting Pass-through HTTPS Listener...
  [2019-09-06 05:07:38,311] [EI-Core]  INFO -
  PassThroughListeningIOReactorManager Pass-through HTTPS Listener
  started on 0.0.0.0:8243 [2019-09-06 05:07:38,311] [EI-Core]  INFO -
  RabbitMQListener RABBITMQ listener started [2019-09-06 05:07:38,320]
  [EI-Core]  INFO - HazelcastClusteringAgent Cluster domain:
  wso2.ei.domain [2019-09-06 05:07:38,320] [EI-Core]  INFO -
  HazelcastClusteringAgent Loading hazelcast configuration from axis2
  clustering configuration [2019-09-06 05:07:38,336] [EI-Core]  INFO -
  HazelcastClusteringAgent Using kubernetes based membership management
  scheme [2019-09-06 05:07:38,352] [EI-Core]  INFO -
  KubernetesMembershipScheme Initializing kubernetes membership
  scheme... [2019-09-06 05:07:38,353] [EI-Core]  INFO -
  ApiBasedPodIpResolver Parameter KUBERNETES_API_SERVER not found,
  checking KUBERNETES_SERVICE_HOST &amp; KUBERNETES_SERVICE_PORT_HTTPS
  [2019-09-06 05:07:38,354] [EI-Core]  INFO - ApiBasedPodIpResolver
  Kubernetes clustering configuration: [api-server]
  <a href="https://10.233.0.1:443" rel="nofollow noreferrer">https://10.233.0.1:443</a> [namespace] wso2 [services]
  wso2ei-integrator-service [skip-master-ssl-verification] true
  [2019-09-06 05:07:38,565] [EI-Core]  INFO - ApiBasedPodIpResolver
  Reading IP addresses from endpoints [2019-09-06 05:07:38,565]
  [EI-Core]  INFO - KubernetesMembershipScheme Member added to cluster
  configuration: [container-ip] 10.233.64.189 [2019-09-06 05:07:38,565]
  [EI-Core]  INFO - KubernetesMembershipScheme Kubernetes membership
  scheme initialized successfully [2019-09-06 05:07:38,567] [EI-Core] 
  INFO - HazelcastClusteringAgent Hazelcast cluster is initializing...
  [2019-09-06 05:07:38,625] [EI-Core] ERROR - DefaultAddressPicker
  [LOCAL] [wso2.ei.domain] [3.5.4] integrator: Name or service not known
  java.net.UnknownHostException: integrator: Name or service not known
    at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)   at
  java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)    at
  java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1277)    at
  java.net.InetAddress.getAllByName(InetAddress.java:1193)  at
  java.net.InetAddress.getAllByName(InetAddress.java:1127)  at
  java.net.InetAddress.getByName(InetAddress.java:1077)     at
  com.hazelcast.instance.DefaultAddressPicker.getPublicAddress(DefaultAddressPicker.java:289)
    at
  com.hazelcast.instance.DefaultAddressPicker.pickAddress(DefaultAddressPicker.java:124)
    at com.hazelcast.instance.Node.(Node.java:143)    at
  com.hazelcast.instance.HazelcastInstanceImpl.(HazelcastInstanceImpl.java:120)
    at
  com.hazelcast.instance.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:152)
    at
  com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:135)
    at
  com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:111)
    at
  com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:58)
    at
  org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent.init(HazelcastClusteringAgent.java:196)
    at
  org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.enableClustering(StartupFinalizerServiceComponent.java:293)
    at
  org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.completeInitialization(StartupFinalizerServiceComponent.java:187)
    at
  org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.serviceChanged(StartupFinalizerServiceComponent.java:317)
    at
  org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at
  org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at
  org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at
  org.wso2.carbon.throttling.agent.internal.ThrottlingAgentServiceComponent.registerThrottlingAgent(ThrottlingAgentServiceComponent.java:123)
    at
  org.wso2.carbon.throttling.agent.internal.ThrottlingAgentServiceComponent.activate(ThrottlingAgentServiceComponent.java:100)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at
  sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)     at
  org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at
  org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at
  org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at
  org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at
  org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at
  org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at
  org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:515)
    at
  org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:220)
    at
  org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:105)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at
  sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)     at
  org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at
  org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at
  org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at
  org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at
  org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at
  org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at
  org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
    at
  org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
    at
  org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
    at
  org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
    at
  org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1230)
    at
  org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1174)
    at
  org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1066)
    at
  org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5433)
    at
  org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5731)
    at
  org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
    at
  org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
    at
  org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)     at
  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at
  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748) [2019-09-06 05:07:38,627]
  [EI-Core] ERROR - StartupFinalizerServiceComponent Cannot initialize
  cluster com.hazelcast.core.HazelcastException:
  java.net.UnknownHostException: integrator: Name or service not known
    at com.hazelcast.util.ExceptionUtil.rethrow(ExceptionUtil.java:67)
    at com.hazelcast.instance.Node.(Node.java:145)    at
  com.hazelcast.instance.HazelcastInstanceImpl.(HazelcastInstanceImpl.java:120)
    at
  com.hazelcast.instance.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:152)
    at
  com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:135)
    at
  com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:111)
    at
  com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:58)
    at
  org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent.init(HazelcastClusteringAgent.java:196)
    at
  org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.enableClustering(StartupFinalizerServiceComponent.java:293)
    at
  org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.completeInitialization(StartupFinalizerServiceComponent.java:187)
    at
  org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.serviceChanged(StartupFinalizerServiceComponent.java:317)
    at
  org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at
  org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at
  org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at
  org.wso2.carbon.throttling.agent.internal.ThrottlingAgentServiceComponent.registerThrottlingAgent(ThrottlingAgentServiceComponent.java:123)
    at
  org.wso2.carbon.throttling.agent.internal.ThrottlingAgentServiceComponent.activate(ThrottlingAgentServiceComponent.java:100)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at
  sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)     at
  org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at
  org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at
  org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at
  org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at
  org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at
  org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at
  org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:515)
    at
  org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:220)
    at
  org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:105)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at
  sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)     at
  org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at
  org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at
  org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at
  org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at
  org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at
  org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at
  org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at
  org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at
  org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at
  org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at
  org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
    at
  org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
    at
  org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
    at
  org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
    at
  org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1230)
    at
  org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1174)
    at
  org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1066)
    at
  org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5433)
    at
  org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5731)
    at
  org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
    at
  org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
    at
  org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)     at
  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at
  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748) Caused by:
  java.net.UnknownHostException: integrator: Name or service not known
    at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)   at
  java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)    at
  java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1277)</p>
</blockquote>

Isuru Gunawardana
<p>You can get this resolved by adding an /etc/host entry to the container. So in K8s you can update the pod template in Deployment as follows.</p>

<blockquote>
<pre><code>spec:
  hostAliases:
    - ip: "127.0.0.1"
      hostnames:
        - "integrator"
</code></pre>
</blockquote>
