Date of Award

Fall 2013

Degree Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Computer Science

First Advisor

Eugene H. Spafford

Second Advisor

Dongyan Xu

Committee Chair

Eugene H. Spafford

Committee Member 1

Dongyan Xu

Committee Member 2

Samuel Liles

Committee Member 3

Sonia Fahmy

Abstract

The vast majority of hosts on the Internet, including mobile clients, are running one of three commodity, general-purpose operating system families. In such operating systems the kernel software executes at the highest processor privilege level. If an adversary is able to hijack the kernel software then by extension he has full control of the system. This control includes the ability to disable protection mechanisms and hide evidence of compromise.

The lack of diversity in commodity, general-purpose operating systems enables attackers to craft a single kernel exploit that has the potential to infect millions of hosts. If enough variants of the vulnerable software exist, then mass exploitation is much more difficult to achieve. We introduce novel kernel diversification techniques to improve kernel security.

Many modern kernels are self-patching; they modify themselves at run-time. Self-patching kernels must therefore allow kernel code to be modified at run-time. To prevent code injection attacks, some operating systems and security mechanisms enforce a W^X memory protection policy for kernel code. This protection policy prevents self-patching kernels from applying patches at run-time. We introduce a novel run-time kernel instruction-level validation technique to validate the integrity of patches at run-time.

Kernels shipped with general-purpose operating systems often contain extraneous code. The code may contain exploitable vulnerabilities or may be pieced together using return/jump-oriented programming to attack the system. Code-injection prevention techniques do not prevent such attacks. We introduce a novel run-time kernel minimization technique to improve kernel security.

We show that it is possible to strengthen the defenses of commodity general-purpose computer operating systems by increasing the diversity of, validating the integrity of, and ensuring the minimality of the included kernel components without modifying the kernel source code. Such protections can therefore be added to existing widely-used unmodified operating systems to prevent malicious software from executing in supervisor mode.

Download

Share

COinS