<p>I'm currently trying the Jenkins kubernetes plugin below, but have some problem.</p>

<p><a href="https://wiki.jenkins-ci.org/display/JENKINS/Kubernetes+Plugin" rel="noreferrer">https://wiki.jenkins-ci.org/display/JENKINS/Kubernetes+Plugin</a></p>

<p>In my case, Jenkins doesn't exist in my kubernetes cluster. This is because I have three kubernetes clusters for dev, staging, and production environments and rather than having three Jenkins service for each env, I want to have one consolidated Jenkins master which operates all three clusters.</p>

<p>Each environment is on an indivisual VPC and Jenkins server is on another VPC, so I setup VPC peering from Jenkins VPC to all other VPCs, and then opened 443 port from Jenkins to k8s master on DEV.</p>

<p>But when I click "Test connection" on "adding new cloud" -> "kubernetes", an error says </p>

<pre><code>javax.servlet.ServletException: io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:796)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
    at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:233)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
    at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:134)
    at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:125)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
    at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
    at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:57)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:418)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:58)
    at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$DescriptorImpl.doTestConnection(KubernetesCloud.java:590)
    at sun.reflect.GeneratedMethodAccessor1736.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:324)
    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:167)
    at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:100)
    at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:124)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
    ... 63 more
Caused by: java.net.NoRouteToHostException: No route to host
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at com.squareup.okhttp.internal.Platform.connectSocket(Platform.java:120)
    at com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:141)
    at com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:112)
    at com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
    at com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
    at com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
    at com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:281)
    at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
    at com.squareup.okhttp.Call.getResponse(Call.java:286)
    at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
    at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
    at com.squareup.okhttp.Call.execute(Call.java:80)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:230)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:416)
    ... 74 more
</code></pre>

<p>I want to make sure if this plugin can be used in my case, because on most examples, it seems Jenkins master is also inside a kubernetes cluster as a k8s pod.</p>

<p>1、prepare a service account for kubernetes-plugin in k8s</p>

<pre><code>apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins
</code></pre>

<p>2、find the according secret generated: jenkins-token-xxxxxx</p>

<p>From the k8s dashboard, we can get the "<strong>ca.crt</strong>" string and "<strong>token</strong>" string from the secret.</p>

<p>3、configure jenkins cloud</p>

<p>In the "<strong>Kubernetes server certificate key</strong>" item, fill in the "<strong>ca.crt</strong>" string.</p>

<p>In the "Credentials" item, creat a credential of type "<strong>Secret text</strong>" or "<strong>OpenShift OAuth token</strong>"(even if we use the pure k8s), fill in the "<strong>token</strong>" string, then use the credential.</p>

<p>"<strong>Test Connection</strong>" should pass now.</p>
