Over the past decade, ARM has accelerated its effort to enable its technology to be leveraged by enterprise users. Today ARM servers are becoming increasingly common, not only with common open source Linux hypervisors (such as KVM and Xen) but also other proprietary solutions. Virtualization has become ubiquitous in enterprise applications, as it allows us to increase efficiency and reduce cost, through the ability to run multiple operating systems (OSes) concurrently on the same hardware resources.
Each OS runs on a virtual machine that operates under the impression of having exclusive access to the processors, peripherals, system memory and IO devices. Recent statistics indicate that enterprise workloads are shifting to the Cloud, which increases the number of VMs being deployed for workload aggregation. The exponential increase of workload deployment to the cloud, demands higher performance throughput and highly scalable memory backplane.
Figure 1: Virtual Machine partitioning with CoreLink CMN-600 and DMC-620 backplane IP
In the enterprise server application space, one physical server is configured to support many virtual machines (VMs). Virtual machines may provide services such as web hosting, cloud services, database and storage management. Having multiple OSes available at any point of time enables live migration of the workload, to the most efficient server, facilitating live update of OSes while the application is running. VMs can be moved when the physical server fails and fault codes may be isolated from other VMs. In addition, server applications often require application specific hardware accelerators that are designed specifically for performing targeted workloads more efficiently. These accelerators, when connected as a network of devices, will provide additional processing power. As such, accelerators which act as system IO devices would need to be ‘virtualized’ to be leveraged by the VMs.
Over the last few years, ARM has dedicated time and resource to creating virtualization whitepapers. In 2010, ARM released our SMMU Whitepaper, which gave an overview of virtualization use cases in the mobile and embedded applications. It also provided technical details of the ARM CoreLink SMMU address translation functionality.
I’m pleased to announce my latest whitepaper is now available for download here - this new whitepaper focusses on the enterprise solution. The whitepaper explains the technical details of IO device virtualization and virtual interrupt management from both hardware and software perspectives. This paper will also provide details of the ARM virtualization architecture, which is best suited for different hypervisor types. In addition, anyone who is interested in the ARM CoreLink SMMU and GIC architecture, should download the whitepaper for more technical insight.
The new whitepaper is expansive, but in case you need some extra encouragement, here are three teasers to what you can learn about in the whitepaper.
The ARM virtualization extension includes additional instructions, system registers, privilege layer through additional Exception Levels (EL) and control enhancements to fulfil virtualization requirements in an ARM system. The ARMv8.0-A virtualization support is most optimal for Type-1 hypervisors, as all Rich OSes are run at the same level of exception in EL1, whilst the hypervisor sits in EL2. This is less optimal for Type-2 hypervisors as EL2 is not tailored for running host OS. ARM v8.1-A adds Virtualization Host Extension (VHE), which introduces shadow copies of EL1 system registers in EL2, enabling the whole Host OS to be run at the same exception level of EL2. This improvement simplifies context switching between host and guest OSes to the same level as Type-1 hypervisors.
Figure 2: Xen (Type-1) and KVM (Type-2) hypervisor
IO virtualization involves virtualizing the IO path between the system and external devices and accelerators. In the infrastructure market, this may apply to server interface adaptors such as the following:
As the number of system devices and device masters increases, the system is bound to be flooded with large amount of address translation requests; ARM SMMU addresses translation capability, supporting different page sizes needed by an enterprise server system and can handle large number of contexts simultaneously. ARM SMMUv3 provides features to complement PCI Express Root Complexes and other potentially large I/O systems. It does this by supporting large numbers of concurrent translation contexts. This includes the memory based configuration for supporting large number of streams.
Figure 3: IO virtualization via ARM CoreLink System Memory Management Unit (SMMU)
The ARM GICv3 architecture includes the following logical components:
With the increasing number of virtual machines, the number of virtual interrupts that need to be moved around the system also increases. The ITS helps to manage the translation of message based interrupts (MSI) more efficiently, by delegating the programming of MSI to guest OSes and subsequently ITS routing them, to the target destination by performing namespace matching in the ITS configuration.
Figure 4: ARM GICv3 architecture
For further details on how ARM technology enables virtualization for enterprise servers, please download the enterprise server virtualization whitepaper below. This whitepaper aims to demonstrate how ARM system IP facilitate IO virtualization with both type-1 and type-2 hypervisor architecture. In addition, ARM has also published the Server Base System Architecture as the foundation of implementing ARM based server SoC.
The link to download the white paper seems to be broken. Can you please check?