Ninja Migration: An Interconnect-Transparent Migration for Heterogeneous Data Centers

A virtual machine (VM) migration is useful for improving flexibility and maintainability in cloud computing environments. However, the heterogeneity of the underlying software and hardware, including CPU and interconnect architectures, makes it hard to migrate a VM. In addition, VM monitor~(VMM)-bypass I/O technologies, which significantly reduce the overhead of I/O virtualization, also make VM migration impossible. Therefore, a VM assigned to an Infiniband device cannot migrate to an Ethernet machine, and vice versa. If we overcome the above barriers, we can increase the potential and possibilities of VM migration. In this paper, we propose an interconnect-transparent migration mechanism to simultaneously migrate multiple co-located VMs between data centers equipped with different interconnect devices. Our implementation of the proposed mechanism, called Ninja migration, is achieved by cooperation between a VMM and an MPI runtime system on the guest OS. We demonstrate fallback and recovery operations on a high performance computing workload using the proposed mechanism. We have confirmed that 1) the proposed mechanism has no performance overhead during normal operations, and 2) MPI processes running on distributed VMs can migrate between an Infiniband cluster and an Ethernet cluster without restarting the processes.