<p>I am new to Docker and Kubernetes, though I have mostly figured out how it all works at this point.</p>

<p>I inherited an app that uses both, as well as KOPS.</p>

<p>One of the last things I am having trouble with is the KOPS setup. I know for absolute certain that Kubernetes is setup via KOPS. There's two KOPS state stores on an S3 bucket (corresponding to a dev and prod cluster respectively)</p>

<p>However while I can find the server that kubectl/kubernetes is running on, absolutely none of the servers I have access to seem to have a kops command.</p>

<p>Am I misunderstanding how KOPS works? Does it not do some sort of dynamic monitoring (would that just be done by ReplicaSet by itself?), but rather just sets a cluster running and it's done?</p>

<p>I can include my cluster.spec or config files, if they're helpful to anyone, but I can't really see how they're super relevant to this question.</p>

<p>I guess I'm just confused - as far as I can tell from my perspective, it <em>looks</em> like KOPS is run once, sets up a cluster, and is done. But then whenever one of my node or master servers goes down, it is self-healing. I would expect that of the node servers, but not the master servers.</p>

<p>This is all on AWS.</p>

<p>Sorry if this is a dumb question, I am just having trouble conceptually understanding what is going on here.</p>

<p><code>kops</code> is a command line tool, you run it from your own machine (or a jumpbox) and it creates clusters for you, it’s not a long-running server itself. It’s like Terraform if you’re familiar with that, but tailored specifically to spinning up Kubernetes clusters.</p>

<p><code>kops</code> creates nodes on AWS via autoscaling groups. It’s this construct (which is an AWS thing) that ensures your nodes come back to the desired number.</p>

<p><code>kops</code> is used for managing Kubernetes clusters themselves, like creating them, scaling, updating, deleting. <code>kubectl</code> is used for managing container workloads that run on Kubernetes. You can create, scale, update, and delete your replica sets with that. How you run workloads on Kubernetes should have nothing to do with how/what tool you (or some cluster admin) use to manage the Kubernetes cluster itself. That is, unless you’re trying to change the “system components” of Kubernetes, like the Kubernetes API or <code>kubedns</code>, which are cluster-admin-level concerns but happen to run on top of Kuberentes as container workloads.</p>

<p>As for how pods get spun up when nodes go down, that’s what Kubernetes as a container orchestrator strives to do. You declare the desired state you want, and the Kubernetes system makes it so. If things crash or fail or disappear, Kubernetes aims to reconcile this difference between actual state and desired state, and schedules desired container workloads to run on available nodes to bring the actual state of the world back in line with your desired state. At a lower level, AWS does similar things — it creates VMs and keeps them running. If Amazon needs to take down a host for maintenance it will figure out how to run your VM (and attach volumes, etc.) elsewhere automatically.</p>
