High Performance Nested Virtualization
An integral part of HVX is a high performance nested hypervisor or Virtual Machine Manager (VMM) capable of running unmodified guests on top of already virtualized hardware. Conventional hypervisors such as VMware ESX™, KVM and Xen are designed to run on physical x86 hardware and use virtualization extensions offered by modern CPUs (Intel VT and AMD SVM) to virtualize the Intel architecture. HVX, on the other hand is a nested hypervisor that runs inside a virtual machine, which utilizes these instruction sets where hardware extensions are available to the hypervisor (currently when running on Oracle Cloud Infrastructure). When these hardware extensions are not available – HVX employs a technology called binary translation to implement high-performance virtualization that does not require these virtualization extensions. Ravello utilizing hardware extensions can deliver higher performance compared to binary translation.
Nested Virtualization Implementations
HVX’s nested virtualization engine supports three modes to offer unparalleled performance when running VMware VMs on cloud. These nested virtualization modes are – hardware assisted, direct on bare metal, and software assisted.
1. Hardware assisted nested virtualization
Oracle Cloud Infrastructure runs on the next generation hardware that supports virtualization extensions. These extensions allow multiple guest operating systems to share the same underlying hardware in safe and efficient manner. HVX utilizes these hardware assist CPU instruction sets to perform its nested virtualization directly on the underlying cloud hardware and offers significant performance improvements over the previous generation of HVX. Typically, the cloud providers do not expose the hardware assisted virtualization extensions to the guest VMs, which limits the performance that customers can realize when operating in a nested virtualization mode. However, with Ravello running on Oracle Cloud Infrastructure, we now have complete access to these hardware assist virtualization extensions, and can make performance boosts a reality.
2. Running directly on bare metal
On Oracle Cloud Infrastructure, HVX also supports the ability to run directly on top of bare metal servers. By eliminating a layer of hypervisor in the middle, HVX is able to provide near native performance.
3. Software assisted nested virtualization
For the underlying clouds (e.g. AWS/GCP) that do not support or expose the hardware assist capabilities for nested virtualization, HVX uses software based nested virtualization technology called binary translation with direct execution to run the VMware VMs on public cloud infrastructure. This technology offers good performance that is acceptable for a wide variety of the workloads.
Nested Virtualization: Performance
Ravello’s HVX will now utilize hardware support for virtualization, wherever these are available. As such, a performance boost of up to 14X can be seen, with a potentially even higher boost when running directly on bare metal compute. With such performance gains, enterprises are now easily able to run their production VMware apps on Oracle Cloud Infrastructure with Ravello.
Also as enterprise needs grow, the apps need to scale to accommodate the demand. With Ravello on Oracle Cloud Infrastructure, enterprises are able to scale their application vertically up to 32 vCPUs per VM, and horizontally across thousands of VMs.
HVX Nested Virtualization: Foundational Technology
The base functionality of HVX nested virtualization is to virtualize the Intel architecture and run unmodified guest operating systems. But because HVX fully controls the execution of its guest VMs, it is also a foundation that enables a lot of advanced features. Below are some examples of current features that are enabled by HVX:
- HVX allows us to intercept all network traffic and therefore allow us to implement our software defined network.
- HVX allows us to intercept all storage access, making our storage overlay possible.
- HVX allows us to run multiple virtual machines inside a single cloud VM. This is called consolidation. Consolidation allows a much higher utilization and in some cases, helps increase performance.
- HVX allows us to virtualize console access: a graphical console for each virtual machine is available.
HVX Nested virtualization: Running VMware or KVM VMs Unmodified on Leading Clouds
In addition to the core nested hypervisor functionality, HVX exposes VMware or KVM devices to the virtual machine running on top. This enables enterprises to run their existing VMware virtual machines unmodified on top of Ravello on top of Oracle Public Cloud. Everything about the VM stays the same - the same operating system, paravirtualized drivers (VMXNet3 network driver, PVSCSI storage driver etc.), application settings, network settings, VMware tools etc.
HVX Nested^2 Virtualization: Run ESXi or KVM on Leading Clouds
Not only can HVX run VMware or KVM virtual machines, but recently we implemented virtualization hardware extensions (Intel VT and AMD V) functionality in software inside HVX. Now HVX can expose a true x86 platform type to the "VM" running on top. This allows enterprises to run hypervisors like KVM and ESXi on top of Ravello on top of Oracle Public Cloud. From an implementation perspective, we have adapted our binary translation so that it recognizes the double-nesting, and effectively removes one layer of nesting and runs the guest directly on top of HVX. As a result, the performance overhead is relatively low. In addition, we have also implemented nested pages support inside HVX which will make running a hypervisor on top of HVX even more efficient.