Optimizing shared memory programs for distributed memory architectures

Seung-Jai Min, Purdue University

Abstract

OpenMP has emerged as the de facto standard for writing parallel programs on shared address space platforms. Programmers can parallelize existing sequential programs in an incremental way with OpenMP directives. In contrast, parallel programming with message-passing for distributed-memory systems is effort intensive. The message-passing approach requires the programmer to parallelize programs as a whole and to explicitly manage data communication between processors. The goal of this dissertation is to extend the high programmer productivity of shared memory programming facilitated by OpenMP to distributed memory architectures, such as clusters. An underlying layer of software Distributed Shared Memory (DSM) is used to enable OpenMP shared memory programs to be run on distributed memory systems. However, the performance of state-of-the-art software DSM systems is impaired by unnecessary communication due to the inefficient mechanisms to detect shared data accesses, especially irregular data accesses. This dissertation introduces the Lean Distributed Shared Memory (LDSM) system to overcome the performance limitations of software DSM systems. LDSM is a thin run-time library layer which is tightly integrated with the compiler. Its region-based shared data access analysis efficiently detects irregular data accesses. Additionally, LDSM includes a run-time overhead reduction technique and communication optimizations. A set of representative regular and irregular benchmarks are used to evaluate the performance of LDSM. The performance results demonstrate that LDSM executes OpenMP applications on distributed memory clusters with a comparable performance to hand-coded message-passing programs.

Degree

Ph.D.

Advisors

Eigenmann, Purdue University.

Subject Area

Electrical engineering

COinS