<p>My goal is to have a kubernetes cluster running with stateless replicas of some kind of frontend which can schedule jobs.</p>
<p>Afterwards my plan is to have multiple runners (as pods) which are polling for scheduled jobs.
Once they receive job data they should launch job executors (also as a pod) on demand.</p>
<p>This should look somehow like this:</p>
<pre><code>  pod A                    pod B
 ________                __________
|        |      kube    |          |
| runner |    ------&gt;   | executor |  
|________|              |__________|
                 .
                 .
  pod A'         .         pod B'
 ________                __________
|        |      kube    |          |
| runner |    ------&gt;   | executor |  
|________|              |__________|
</code></pre>
<p>Basically I am kind of inspired by the gitlab architecture.
<a href="https://docs.gitlab.com/runner/executors/kubernetes.html#kubernetes-executor-interaction-diagram" rel="nofollow noreferrer">https://docs.gitlab.com/runner/executors/kubernetes.html#kubernetes-executor-interaction-diagram</a></p>
<p>Therefore it would be really awesome if you could give me a hint where to start with.
I was considering to use the python api of the cluster but in this I have to somehow make the cluster administration available to the runner.</p>
<p>So I'm really thankful for any idea or approach how to realize such an architecture of pod lifecycling.</p>
<p>Do you see any security risks with such an approach or is this reasonable securitywise?</p>
<p>Thank you very much for your help and hints</p>

<p>Here is a <a href="https://kubernetes.io/docs/reference/using-api/client-libraries/" rel="nofollow noreferrer">list of available kubernetes api clients</a>. You will use a <strong><a href="https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/" rel="nofollow noreferrer">service account</a></strong> to make authorized calls with your api client against the <strong>kube-apiserver</strong> running in the <strong>controlplane</strong> node. I made a first idea draft for your architecture. Maybe that is inspiring to you. Good look with your project! Alternative to the database you may want to use a message queue.</p>
<p><a href="https://i.stack.imgur.com/SFBOV.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/SFBOV.png" alt="enter image description here" /></a></p>
