Efficient, Dynamic Mechanisms for Ordering Data Accesses and Management Operations in the Cloud

Tariq Mahmood, Purdue University

Abstract

The loosely-coupled and distributed nature of modern cloud-based systems creates two major challenges in (i) ordering data operations (i.e., reads and writes to state maintained in cloud storage), and (ii) orchestrating and ordering management operations (i.e., complex coordinated actions that are required across administrative domains). For ordering data accesses, recent proposals advocate for causal consistency as it offers intuitive ordering while preserving availability under failures and network partitions. However, such systems have seen limited deployment in practice as they employ full replication of all the data in each data center (DC), and static binding for clients. Full replication incurs high cost, and static binding limits the availability of the system. We present Karma, a system that offers efficient, dynamic mechanisms to enforce causal order without requiring full replication of data at each DC. Our evaluation shows that Karma (i) incurs much lower cost than a fully-replicated causal store; and (ii) offers higher availability and better performance than naive partial-replication-based systems at similar costs. For management operations, it is challenging to track the full impact of one change initiated by one cloud service as its effect propagates across other layered, loosely-coupled, and independently administered cloud services. The current approach to handling such changes is typically manual, error-prone, and time consuming. We present a novel middleware for Automated Change COoRDination (ACCORD) across cloud services. ACCORD provides abstractions that allow cloud services to explicitly register direct dependencies on shared resources, perform changes to these resources, and specify individual policies to react to changes to these resources. At the core of our design is a novel change coordination protocol that identifies all dependent entities (including transitive dependencies across all affected cloud services), and executes the actions that each entity takes in response to the change. The protocol orchestrates each change as a notify-on-failure transaction, offering atomic semantics when the transaction is successful, and notifying stakeholders (so they can take appropriate action) when it fails.

Degree

Ph.D.

Advisors

Thottethodi, Purdue University.

Subject Area

Computer Engineering|Computer science

COinS