<p>I'd like to deploy kubernetes on a large physical server (24 cores) and I'm uncertain as to a number of things.</p>

<p>What are the pros and cons of creating virtual machines for the k8s cluster other than running on bare-metal.</p>

<p>I have the following considerations:</p>

<ul>
<li>Creating vms will allow for work load isolation. New vms for experiments can be created and assigned to devs.</li>
<li>On the other hand, with k8s running on bare metal a new NAMESPACE can be created for each developer for experimentation and they can run their code in it. After all their code should be running in docker containers.</li>
</ul>

<p><strong>Security:</strong></p>

<ul>
<li>Having vms would limit the amount of access given to future maintainers, limiting the amount of damage that could be done. While on the other hand the primary task for any future maintainers would be adding/deleting nodes and they would require bare metal access to do that.</li>
</ul>

<p><strong>Authentication:</strong></p>

<ul>
<li>At the moment devs would only touch the server when their code runs through the CI pipeline and their running deployments are deployed. But what about viewing logs? Could we setup tiered kubectl authentication to allow devs to only access whatever namespaces have been assigned to them (I believe this should be possible with the k8s namespace authorization plugin).</li>
</ul>

<p>A number of vms already exist on the server. Would this be an issue?</p>

<p>128 cores and doubts.... That is a lot of cores for a single server.</p>

<p>For kubernetes however this is not relevant:
Kubernetes can use different sized servers and utilize them to the maximum. However if you combine the master server processes and the node/worker processes on a single server, you might create unwanted resource issues. You can manage those with namespaces, as you already mention.</p>

<p>What we do is use continuous integration with namespaces in a single dev/qa kubernetes environment in which changes have their own namespace (So we run many many namespaces) and run full environment deployments in those namespaces. A bunch of shell scripts are used to manage this. This works both with a large server as what you have, as well as it does with smaller (or virtual) boxes. The benefit of virtualization for you could mainly be in splitting the large box in smaller ones so that you can also use it for other purposes then just kubernetes (yes, kubernetes runs except MS Windows, no desktops, no kernel modules for VPN purposes, etc).</p>
