<p>I've been experimenting with the fabric8 events() API in my attempts to get  a working solution for <a href="https://stackoverflow.com/questions/37020369/best-way-in-kubernetes-to-receive-notifications-of-errors-when-instantiating-res">this question</a> that I asked a while back.</p>

<p>The answer to the aforementioned question was "use the events() API !"  which seems like an excellent idea... However, I am running into a strange error when I try to use that API in even the most basic way I can imagine. I have included (in the section SAMPLE PROGRAM)  a simple program  which illustrates the problem. Below are excerpts of the 
code which are most relevant:  </p>

<pre><code>  val eventWatcher: Watcher[Event] = new Watcher[Event]() {
    def eventReceived(action: Action, event: Event) {
      logger.info("notified!")
    }
  }

   ... 
  kube = getConnection  // get client connection to k8s server
  kube.events().watch(eventWatcher)
</code></pre>

<p>After I set up the event watch I create a pod.  The pod creates just fine.  However, when the 'eventReceived' call back is being invoked I get the 
error and stack trace that I list below in ERRORS. I never get into the eventReceived method (and the string 'notified' never appears in the logs).</p>

<p>Any tips, suggestions or guidance, greatly appreciated !
 - chris</p>

<p>SAMPLE PROGRAM</p>

<pre><code>import com.fasterxml.jackson.databind.ObjectMapper
import scala.collection.JavaConverters._
import com.ning.http.client.ws.WebSocket
import com.typesafe.scalalogging.StrictLogging
import io.fabric8.kubernetes.api.model._
import io.fabric8.kubernetes.client.DefaultKubernetesClient.ConfigBuilder
import io.fabric8.kubernetes.client.Watcher.Action
import io.fabric8.kubernetes.client.dsl.Resource
import io.fabric8.kubernetes.client.{DefaultKubernetesClient, Watcher}

object ErrorTest extends App with StrictLogging {
  // corresponds to --insecure-skip-tls-verify=true, according to io.fabric8.kubernetes.api.model.Cluster
  val trustCerts = true
  val k8sUrl = "http://localhost:8080"
  val namespaceName = "default" // replace this with name of a namespace that you know exists
  val imageName: String = "nginx" //  make image name to load to pod a variable so we can experiment with err conditions

  val eventWatcher: Watcher[Event] = new Watcher[Event]() {
    def eventReceived(action: Action, event: Event) {
      logger.info("notified!")
    }
  }

  def go(): Unit = {
    val kube = getConnection
    dumpNamespaces(kube)
    kube.events().watch(eventWatcher)
    deployPodWithWatch(kube, getPod(image = imageName))
  }

  def deployPodWithWatch(kube: DefaultKubernetesClient, pod: Pod): Unit = {
    kube.pods().inNamespace(namespaceName).create(pod) /*  create the pod ! */
  }

  def getPod(image: String): Pod = {
    val jsonTemplate =
      """
    |{
    | "kind": "Pod",
    | "apiVersion": "v1",
    | "metadata": {
    |   "name": "podboy",
    |   "labels": {
    |     "app": "nginx"
    |   }
    | },
    | "spec": {
    |   "containers": [
    |     {
    |     "name": "podboy",
    |     "image": "&lt;image&gt;",
    |     "ports": [
    |       {
    |         "containerPort": 80,
    |         "protocol": "TCP"
    |       }
    |     ]
    |     }
    |   ]
    | }
    |}
      """.
    stripMargin
    val replacement: String = "image\": \"" + image
    val json = jsonTemplate.replaceAll("image\": \"&lt;image&gt;", replacement)
    System.out.println("json:" + json);
    new ObjectMapper().readValue(json, classOf[Pod])
  }

  def dumpNamespaces(kube: DefaultKubernetesClient): Unit = {
    val namespaceNames = kube.namespaces().list().getItems.asScala.map {
      (ns: Namespace) =&gt; {
    ns.getMetadata.getName
      }
    }
    System.out.println("namespaces are:" + namespaceNames);
  }

  def getConnection = {
    val configBuilder = new ConfigBuilder()
    val config =
      configBuilder.
    trustCerts(trustCerts).
    masterUrl(k8sUrl).
    build()
    new DefaultKubernetesClient(config)
  }

  go()
}
</code></pre>

<p>ERRORS </p>

<pre><code>    22:03:16.656 [New I/O worker #2] ERROR i.f.k.c.dsl.internal.BaseOperation$3 - Could not deserialize watch event: {"type":"MODIFIED","object":{"kind":"Event","apiVersion":"v1","metadata":{"name":"spark-master-rc-zlxpp.144f898e0549644b","namespace":"dummyowner-workflow-dda91220-b63-1544803905","selfLink":"/api/v1/namespaces/dummyowner-workflow-dda91220-b63-1544803905/events/spark-master-rc-zlxpp.144f898e0549644b","resourceVersion":"204015294","creationTimestamp":null,"deletionTimestamp":"2016-05-18T06:03:06Z"},"involvedObject":{"kind":"Pod","namespace":"dummyowner-workflow-dda91220-b63-1544803905","name":"spark-master-rc-zlxpp","uid":"d413698c-19c7-11e6-9f26-74dbd1a09231","apiVersion":"v1","resourceVersion":"203578210","fieldPath":"spec.containers{spark-master}"},"reason":"Failed","message":"Failed to pull image \"ecr.vip.ebayc3.com/krylov/spark-1.5.1:${env}\": image pull failed for ecr.vip.ebayc3.com/krylov/spark-1.5.1:${env}, this may be because there are no credentials on this request.  details: (Tag ${env} not found in repository ecr.vip.ebayc3.com/krylov/spark-1.5.1)","source":{"component":"kubelet","host":"kubernetes-minion-105-4040.slc01.dev.ebayc3.com"},"firstTimestamp":"2016-05-18T02:47:26Z","lastTimestamp":"2016-05-18T05:03:05Z","count":815}}
    com.fasterxml.jackson.databind.JsonMappingException: No resource type found for kind:Event
     at [Source: {"type":"MODIFIED","object":{"kind":"Event","apiVersion":"v1","metadata":{"name":"spark-master-rc-zlxpp.144f898e0549644b","namespace":"dummyowner-workflow-dda91220-b63-1544803905","selfLink":"/api/v1/namespaces/dummyowner-workflow-dda91220-b63-1544803905/events/spark-master-rc-zlxpp.144f898e0549644b","resourceVersion":"204015294","creationTimestamp":null,"deletionTimestamp":"2016-05-18T06:03:06Z"},"involvedObject":{"kind":"Pod","namespace":"dummyowner-workflow-dda91220-b63-1544803905","name":"spark-master-rc-zlxpp","uid":"d413698c-19c7-11e6-9f26-74dbd1a09231","apiVersion":"v1","resourceVersion":"203578210","fieldPath":"spec.containers{spark-master}"},"reason":"Failed","message":"Failed to pull image \"ecr.vip.ebayc3.com/krylov/spark-1.5.1:${env}\": image pull failed for ecr.vip.ebayc3.com/krylov/spark-1.5.1:${env}, this may be because there are no credentials on this request.  details: (Tag ${env} not found in repository ecr.vip.ebayc3.com/krylov/spark-1.5.1)","source":{"component":"kubelet","host":"kubernetes-minion-105-4040.slc01.dev.ebayc3.com"},"firstTimestamp":"2016-05-18T02:47:26Z","lastTimestamp":"2016-05-18T05:03:05Z","count":815}}; line: 1, column: 1156] (through reference chain: io.fabric8.kubernetes.api.model.WatchEvent["object"])
            at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:164) ~[jackson-databind-2.4.1.jar:2.4.1]
            at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:757) ~[jackson-databind-2.4.1.jar:2.4.1]
            at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:41) ~[kubernetes-model-1.0.3.jar:1.0.3]
            at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:29) ~[kubernetes-model-1.0.3.jar:1.0.3]
            at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538) ~[jackson-databind-2.4.1.jar:2.4.1]
            at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:99) ~[jackson-databind-2.4.1.jar:2.4.1]
            at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:242) ~[jackson-databind-2.4.1.jar:2.4.1]
            at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:118) ~[jackson-databind-2.4.1.jar:2.4.1]
            at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1269) ~[jackson-databind-2.4.1.jar:2.4.1]
            at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:896) ~[jackson-databind-2.4.1.jar:2.4.1]
            at io.fabric8.kubernetes.client.dsl.internal.BaseOperation$3.onMessage(BaseOperation.java:422) ~[kubernetes-client-1.2.2.jar:na]
            at com.ning.http.client.providers.netty.ws.NettyWebSocket.notifyTextListeners(NettyWebSocket.java:240) [async-http-client-1.9.29.jar:na]
            at com.ning.http.client.providers.netty.ws.NettyWebSocket.onTextFragment(NettyWebSocket.java:281) [async-http-client-1.9.29.jar:na]
            at com.ning.http.client.providers.netty.handler.WebSocketProtocol.handle(WebSocketProtocol.java:162) [async-http-client-1.9.29.jar:na]
            at com.ning.http.client.providers.netty.handler.Processor.messageReceived(Processor.java:88) [async-http-client-1.9.29.jar:na]
            at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:68) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.10.3.Final.jar:na]
            at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.10.3.Final.jar:na]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
            at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
</code></pre>

<p>Looking at the stack trace indicates your using an older version of the fabric8 kubernetes client (we migrated to okhttp from async-http-client a while back). First you need to upgrade to the latest version (<code>io.fabric8:kubernetes-client:1.3.90</code> at time of writing). Here's a snippet for watching events:</p>

<pre><code>Watch watch = client.events().inAnyNamespace().watch(new Watcher&lt;Event&gt;() {
    @Override
    public void eventReceived(Action action, Event resource) {
        logger.info("{}: {}", action, resource);
    }

    @Override
    public void onClose(KubernetesClientException e) {
        if (e != null) {
            e.printStackTrace();
            logger.error(e.getMessage(), e);
        }
    }
});
</code></pre>

<p>Unless you want to watch  the namespace you want to watch with <code>client.events().inNamespace(...).watch(...)</code>.</p>
