<p>I am trying to use KubeVirt with GKE cluster. </p>

<p>I found I am able to create a nested virtualization enabled GCP VM, but I didn't find a way to achieve the same thing for GKE cluster node.</p>

<p>If I cannot enable nested virtualization for GKE cluster node, I can only use the kubevirt with debug.useEmulation which is not what I want.</p>

<p>Thanks</p>

chen lin
<p>Yes you can -- it isn't even hard to do, it just isn't very intuitive.</p>
<ol>
<li><p>Start a GKE cluster with ubuntu/containerd, n1-standard nodes and minimum cpu of Haswell.  I think you also need to enable &quot;Basic Authorization&quot; to get virtctl working (sorry).</p>
</li>
<li><p>Find the template used for your new cluster, then to determine the proper source image:</p>
<p>gcloud compute instance-templates describe  --format=json | jq &quot;.properties.disks[0].initializeParams.sourceImage&quot;</p>
</li>
<li><p>Create a copy of the source disk with nested virtualization enabled:</p>
<p>gcloud compute images --project $PROJECT create $NEW_IMAGE_NAME --source-image $SOURCE_IMAGE --source-image-project=$SOURCE_PROJECT --licenses &quot;https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx&quot;</p>
</li>
<li><p>Use &quot;Create Similar&quot; on the template for your GKE cluster.  Change the boot disk to $NEW_IMAGE_NAME.  You will also need to drill down to networking/alias and change the default subnet to your pod network.</p>
</li>
<li><p>Trigger a rolling update on the group for your GKE nodes to move them to the new template.</p>
</li>
<li><p>You can now install kubevirt (I had to use 0.38.1 instead of the current)</p>
</li>
</ol>
<p>Caveats: I don't know how to use google disk images for kubevirt which would be an obvious match.  I haven't even figured out how to get private GCR working with CDI.  Oh, and console doesn't work due to websocket problems.  But... you can shell to a gke node and see /dev/kvm, you can also kubevirt a VM then ssh into it, so yes, it does work.</p>
<p>Anyone know how to make any of this better?</p>
