<p>This question is somewhat related to one of <a href="https://stackoverflow.com/questions/57043048/best-practices-when-implementing-ci-cd-pipeline-using-github-jenkins-kubernetes">my previous questions</a> as in it gives a clearer idea on what I am trying to achieve.. This question is about an issue I ran into when trying to achieve the task in that previous question...</p>

<p>I am trying to test if my <code>kubectl</code> works from within the Jenkins container. When I start up my Jenkins container I use the following command:</p>

<pre><code>docker run \ 
    -v /home/student/Desktop/jenkins_home:/var/jenkins_home \
    -v $(which kubectl):/usr/local/bin/kubectl \ #bind docker host binary to docker container binary
    -v ~/.kube:/home/jenkins/.kube \ #docker host kube config file stored in /.kube directory. Binding this to $HOME/.kube in the docker container
    -v /var/run/docker.sock:/var/run/docker.sock \ 
    -v $(which docker):/usr/bin/docker -v ~/.kube:/home/root/.kube \ 
    --group-add 998 
    -p 8080:8080 -p 50000:50000 
    -d --name jenkins jenkins/jenkins:lts
</code></pre>

<p>The container starts up and I can login/create jobs/run pipeline scripts all no issue.</p>

<p>I created a pipeline script just to check if I can access my cluster like this:</p>

<pre><code>pipeline {
    agent any
    stages {
        stage('Kubernetes test') {
            steps {
                sh "kubectl cluster-info"
            }
        }
    }
}
</code></pre>

<p>When running this job, it fails with the following error:</p>

<pre><code>+ kubectl cluster-info // this is the step

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
error: the server doesn't have a resource type "services"
</code></pre>

<p>Thanks!</p>

<p>I'm not getting why there is:</p>

<p><code>-v $(which kubectl):/usr/local/bin/kubectl -v ~/.kube:/home/jenkins/.kube</code></p>

<p><code>/usr/local/bin/kubectl</code> is a kubectl binary and <code>~/.kube:/home/jenkins/.kube</code> should be the location where the kubectl binary looks for the cluster context file i.e. <code>kubeconfig</code>. First, you should make sure that the <code>kubeconfig</code> is mounted to the container at <code>/home/jenkins/.kube</code> and is accessible to <code>kubectl</code> binary. After appropriate volume mounts, you can verify by creating a session in the jenkins container with <code>docker container exec -it jenkins /bin/bash</code> and test with <code>kubectl get svc</code>. Make sure you have <code>KUBECONFIG</code> env var set in the session with:</p>

<pre><code>export KUBECONFIG=/home/jenkins/.kube/kubeconfig
</code></pre>

<p>Before you run the verification test and</p>

<pre><code>withEnv(["KUBECONFIG=$HOME/.kube/kubeconfig"]) {
// Your stuff here
}
</code></pre>

<p>In your pipeline code. If it works with the session, it should work in the pipeline as well.</p>

<p>I would personally recommend to create a custom Docker image for Jenkins which will contain <code>kubectl</code> binary and other utilities necessary (such as <code>aws-iam-authenticator</code> for AWS EKS IAM-based authentication) for working with Kubernetes cluster. This creates isolation between your host system binaries and your Jenkins binaries. </p>

<p>Below is the <code>Dockerfile</code> I'm using which contains, <code>helm</code>, <code>kubectl</code> and <code>aws-iam-authenticator</code>.</p>

<pre><code># This Dockerfile contains Helm, Docker client-only, aws-iam-authenticator, kubectl with Jenkins LTS.

FROM jenkins/jenkins:lts
USER root

ENV VERSION v2.9.1
ENV FILENAME helm-${VERSION}-linux-amd64.tar.gz
ENV HELM_URL https://storage.googleapis.com/kubernetes-helm/${FILENAME}
ENV KUBE_LATEST_VERSION="v1.11.0"

# Install the latest Docker CE binaries
RUN apt-get update &amp;&amp; \
    apt-get -y install apt-transport-https \
      ca-certificates \
      curl \
      gnupg2 \
      software-properties-common &amp;&amp; \
    curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg &gt; /tmp/dkey; apt-key add /tmp/dkey &amp;&amp; \
    add-apt-repository \
      "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
      $(lsb_release -cs) \
      stable" &amp;&amp; \
   apt-get update &amp;&amp; \
   apt-get -y install docker-ce \
   &amp;&amp; curl -o /tmp/$FILENAME ${HELM_URL} \
   &amp;&amp; tar -zxvf /tmp/${FILENAME} -C /tmp \
   &amp;&amp; mv /tmp/linux-amd64/helm /bin/helm \
   &amp;&amp; rm -rf /tmp/linux-amd64/helm \
   &amp;&amp; curl -L https://storage.googleapis.com/kubernetes-release/release/${KUBE_LATEST_VERSION}/bin/linux/amd64/kubectl -o /usr/local/bin/kubectl \
   &amp;&amp; chmod +x /usr/local/bin/kubectl \
   &amp;&amp; curl -L https://amazon-eks.s3-us-west-2.amazonaws.com/1.11.5/2018-12-06/bin/linux/amd64/aws-iam-authenticator -o /usr/local/bin/aws-iam-authenticator \
   &amp;&amp; chmod +x /usr/local/bin/aws-iam-authenticator
</code></pre>
