Protecting Kernel Code and Data with a Virtualization-Aware Collaborative Operating System

The traditional virtual machine usage model advocates placing security mechanisms in a trusted VM layer and letting the untrusted guest OS run unaware of the presence of virtualization. In this work we challenge this traditional model and propose a collaboration approach between a virtualization-aware operating system and a VM layer to prevent tampering against kernel code and data. Our integrity model is a relaxed version of Biba's and the main idea is to have all attempted writes into kernel code and data segments checked for validity at VM level. The OS-VM collaboration bridges the semantic gap between tracing low integrity objects at OS-level (files, processes, modules, allocated areas) and architecture-level (memory and registers). We have implemented this approach in a proof-of-concept prototype and have successfully tested it against 6 rootkits (including a non-control data attack) and 4 real-world benign LKM/drivers. All rootkits were prevented from corrupting kernel space and no false positive was triggered for benign modules. Performance measurements show that the average overhead to the VM for the OS-VM communication is low (7%, CPU benchmarks). The greatest overhead is caused by the memory monitoring module inside the VM: 1.38X alone and 1.46X when combined with the OS-VM communication. For OS microbenchmarks the slowdown for the OS-VM communication was 1.16X on average.