Virtualization for Network-based Multicore Telecommunication systems – System Requirements

October 29th, 2009

By Alexander Bachmutsky – Chief Architect at Nokia Siemens Networks

Virtualization brings a number of system requirements to make the technology integration a viable option:

Complete VM isolation for memory, caches, queues, buffer pools and other optionally shared on- or off-chip resources. It is possible to implement some of the isolation cases using software techniques, but the performance impact becomes usually prohibitive.

Additional hypervisor prioritization level. Similarly to the above paragraph, it is possible to live without it, but it would make the hypervisor vulnerable to malfunctioned kernel.

Less than 5% (1%-2% is highly desirable) of the performance overhead compared to similar configuration without the hypervisor. This is one of the most critical requirements and potentially a red flag for many existing hypervisors designed mainly for data center applications.

When a number of cores is small, there is a need for core virtualization, which means running multiple VMs on the same core or hardware thread. However, that need becomes less and less critical with number of cores growing. For example, in multicore chips with many cores this requirement is not very strong, because each core represents relatively low portion of the entire processing capacity: in 16-core Cavium Octeon Plus family it is about 6%, dropping to only slightly more than 3% in recently announced 32-core Cavium Octeon-II family and about 1.5% in 64-core Tilera processor. The fact that with growing number of cores it is possible to simplify the virtualization is very crucial for multicore systems to achieve a viable solution and performance. It is important to mention that we do not believe at all in running VM per hardware thread. It works, of course, but it is very inefficient practically on all existing hardware multi-threaded cores because of limited TLBs, caches and other hardware resources per thread.

Need for I/O virtualization with channel and throughput enforcement. It can be using VLANs for Ethernet, subchannels for SPI3, SPI4, latest PCI Express, and proprietary extensions for Broadcom and Fulcrum Ethernet ports. The minimum guaranteed throughput has to be available to any channel when needed. This requirement includes also DMA virtualization, because in most architectures number of available DMA channels can be smaller than a number of VMs in the system.

Need for shared resource virtualization. It includes shared caches, shared hardware offload engines and other on- and off-chip hardware blocks.

Compact hypervisor layer. Independently on the hypervisor architecture being either microkernel-like layer running on one or even every core or being just a library used for system calls, the hypervisor code has to be small. This requirement relies on well-known statistics that the module stability is proportional to its code size. Simply speaking, larger the code, more bugs it will have. At the same time, the hypervisor becomes a single point of failure in the system, and the hypervisor failure cannot be tolerated in highly available telecommunication products.

There is a need for standardized mechanism of VM fault detection together with VM load/unload capabilities.

Efficient inter-VM communication is a must. There are many ways to implement this communication using software or hardware blocks, but throughput, latency and Quality-of-Service awareness are critical factors for every implementation.

Dynamic traffic mapping between VMs. If VMs are used to separate processing of data, control and management planes, there is a need to separate the traffic, and frequently it cannot be done based on physical ports. It becomes even more complex if multiple instances (VMs) for the same, for example, data plane are running simultaneously. Sych separation can be implemented either in hardware classifier before the packet is being scheduled to a particular VM, or a dedicated VM (which can cause a bottleneck in the system), or the hypervisor code itself (making hypervisor code larger and product-specific).

Rating: 9.5/10 (2 votes cast)

Rating: +2 (from 2 votes)