Public clouds are amazing platforms. They are ideally suited for a variety of workloads. However, one cannot run hypervisors such as KVM or VMware ESXi™ on Public Clouds. This is unfortunate, because the public cloud is an ideal platform for development/testing, demos, PoCs and training environments. This basically locks out the VMware and OpenStack/KVM ecosystem from being able to effectively use on-demand public cloud resources. However, with Ravello’s nested virtualization technology, it is possible to run ESXi (and KVM) on the cloud - with “hardware acceleration”. This opens up a whole host of use-cases for the virtualization (and specifically VMware) ecosystem.

Why one cannot typically run ESXi in the cloud

The reason one cannot typically run a hypervisor in the cloud is that modern hypervisors such as KVM and ESXi need virtualization extensions in silicon (Intel VT/ AMD-V) in order to run. These extensions are not available (or visible) in a cloud VM.

In order to be able to run ESXi natively on Public cloud, the respective virtualization platforms (Xen in the case of AWS and KVM in the case of Google) need to support a “nested virtualization” feature. You can think of this as the hypervisor passing through VT access to the VM. In the meantime, the VMware and KVM/OpenStack ecosystems are stuck. Ravello solves this problem today.

HVX: Virtual infrastructure for the cloud - nested virtualization

At Ravello, we have developed an infrastructure technology we call HVX. One element of HVX is a high performance nested hypervisor that is designed to run on already virtualized hardware (so essentially, in a cloud VM). HVX implements binary translation with direct execution, and exposes VMware or KVM devices (in the form of a platform type) to a VM running on top. As a result, enterprises can take their VMware VMs and run them on Public cloud without making any changes. The VMware VM thinks its running on ESXi but instead its running on HVX.

Ravello's nested hypervisor enables running VMware / KVM workloads on AWS or Google without modification


The main use-case for this deployment mode is for enterprises to be able to take existing VMware workloads (VMware VMs along with their networking and storage) and run them in AWS or Google cloud without making any changes. This is ideal for development, test, demo and training environments for applications such as Microsoft SharePoint, Exchange, Oracle, SAP, and network and security appliances from F5, CheckPoint, Juniper and others.

Nested ^2 virtualization: Running ESXi in Public Cloud

Running VMware VMs in Public cloud is one thing. Running the ESXi hypervisor is something else. In order to tackle the latter, Ravello implemented Intel VT/AMD-V technology along with Nested Page Tables (NPT) in software in HVX. This makes a cloud VM look like real physical hardware (a modern server). Since virtualization extensions are now visible in the cloud VM, one can run a hypervisor (designed to run on physical servers) such as ESXi in the cloud VM. The diagram below illustrates the 2 levels of nesting (hence nested ^2).

Through exposing Intel-VT, Ravello's nested virtualization engine enable ESXi to run in the public cloud

As mentioned earlier, HVX does binary translation for kernel code for the nested guest. This is exactly how its also done for the nested^2 guest - except that it is one additional context switch. With this implementation, the guest instructions are executed directly on the physical CPU.


We believe that this mode of operation has some tremendous implications for the VMware ecosystem. VMware technology alliance partners (TAP), resellers and solutions provider partners, training partners and enterprise customers - can spin up complete ESXi environments on-demand in Public cloud for development, test, training, demos and PoCs - and pay simply for usage. This capability radically changes their economics as they no longer need to have on-premises lab hardware infrastructure for most use-cases. Spinning up complex ESXi environments in the cloud can cost as little as $0.14 per hour for 2 vCPU/ 4GB RAM chunks.


