<p>I am deploying <a href="https://docs.confluent.io/current/connect/index.html" rel="nofollow noreferrer">Kafka-connect</a> on Google Kubernetes Engine (GKE) using <a href="https://github.com/confluentinc/cp-helm-charts/tree/master/charts/cp-kafka-connect" rel="nofollow noreferrer">cp-kafka-connect</a> Helm chart in distributed mode. </p>

<p>A working Kafka cluster with broker and zookeeper is already running on the same GKE cluster. I understand I can create connectors by sending post requests to <code>http://localhost:8083/connectors</code> endpoint once it is available.
However, Kafka-connect container goes into RUNNING state and then starts loading the jar files and till all the jar files are loaded the endpoint mentioned above is unreachable.</p>

<p>I am looking for a way to automate the step of manually <code>exec</code> the pod, check if the endpoint is ready and then send the post requests. I have a shell script that has a bunch of <code>curl -X POST</code> requests to this endpoint to create the connectors and also have config files for these connectors which work fine with standalone mode (using Confluent platform show in this <a href="https://www.confluent.io/blog/simplest-useful-kafka-connect-data-pipeline-world-thereabouts-part-1/" rel="nofollow noreferrer">confluent blog</a>).</p>

<p>Now there are only two ways to create the connector:</p>

<ol>
<li>Somehow identify when the container is actually ready (when the endpoint has started listening) and then run the shell script containing the curl requests</li>
<li>OR use the configuration files as we do in standalone mode (Example: <code>$ &lt;path/to/CLI&gt;/confluent local load connector_name -- -d /connector-config.json</code>)</li>
</ol>

<p>Which of the above approach is better?</p>

<p>Is the second approach (config files) even doable with distributed mode?</p>

<ul>
<li>If <strong><em>YES:</em></strong> How to do that?</li>
<li>If <strong><em>NO:</em></strong> How to successfully do what is explained in the first approach?</li>
</ul>

<p><strong><em>EDIT:</em></strong>
With reference to his <a href="https://github.com/confluentinc/cp-docker-images/issues/467" rel="nofollow noreferrer">github issue</a>(thanks to @cricket_007's answer below)  I added the following as the container command and connectors got created after the endpoint gets ready:</p>

<pre><code>...
command:
  - /bin/bash
  - -c
  - |
    /etc/confluent/docker/run &amp;
    echo "Waiting for Kafka Connect to start listening on kafka-connect  "
    while : ; do
      curl_status=`curl -s -o /dev/null -w %{http_code} http://localhost:8083/connectors`
      echo -e `date` " Kafka Connect listener HTTP state: " $curl_status " (waiting for 200)"
      if [ $curl_status -eq 200 ] ; then
        break
      fi
      sleep 5
    done
    echo -e "\n--\n+&gt; Creating Kafka Connector(s)"
    /tmp/scripts/create-connectors.sh
    sleep infinity
...
</code></pre>

<p><code>/tmp/scripts/create-connectors.sh</code> is a script mounted externally containing a bunch of POST requests using CURL to the Kafka-connect API.</p>

Amit Yadav
<p><code>confluent local</code> doesn't interact with a remote Connect cluster, such as one in Kubernetes.</p>

<p>Please refer to the <a href="https://kafka.apache.org/documentation/#connect_rest" rel="nofollow noreferrer">Kafka Connect REST API</a></p>

<p>You'd connect to it like any other RESTful api running in the cluster (via a Nodeport, or an Ingress/API Gateway for example) </p>

<blockquote>
  <p>the endpoint mentioned above is unreachable.</p>
</blockquote>

<p>Localhost is the physical machine you're typing the commands into, not the remote GKE cluster</p>

<blockquote>
  <p>Somehow identify when the container is actually ready</p>
</blockquote>

<p>Kubernetes health checks are responsible for that </p>

<pre><code>kubectl get services
</code></pre>

<blockquote>
  <p>there are only two ways to create the connector</p>
</blockquote>

<p>That's not true. You could additional run Landoop's Kafka Connect UI or Confluent Control Center in your cluster to point and click. </p>

<p>But if you have local config files, you could also write code to interact with the API</p>

<p>Or try and see if you can make a PR for this issue </p>

<p><a href="https://github.com/confluentinc/cp-docker-images/issues/467" rel="nofollow noreferrer">https://github.com/confluentinc/cp-docker-images/issues/467</a></p>
