Asynchronous parallel and distributed computing: Theoretical modeling and experimental evaluation

Longsong Lin, Purdue University

Abstract

A methodology is introduced for minimizing the total execution time for a class of large-scale parallel iterative algorithms. The basic idea centers around the concept of balancing the amount of time each processor spends doing computation with the amount of time spent waiting for information from other processors. The performance of the proposed methodology is evaluated analytically and timing results from the nCUBE 2 supercomputer are recorded to justify the theoretical analysis. Features of the assumed architecture and the structure of the iterative problem to be solved are captured using stochastic modeling techniques. An important control parameter in the methodology, called the degree of synchronization, characterizes the fraction of time each processor waits for new information from other processors before starting a new phase of computation. Both theoretical and experimental studies are done to characterize the relationship between convergence time and the degree of synchronization. The results indicate that significant improvement in convergence time can be achieved by proper control of the parameters in the proposed model. For the case of tightly-coupled iterative equations, it is preferable to select a relatively high degree of synchronization; for loosely-coupled equations, it is preferable to select a relatively low degree of synchronization. The machine granularity is also shown to have a strong impact on determining the optimal value of the degree of synchronization. Various levels of machine granularity are emulated and timings are recorded from the nCUBE 2. Because parallel iterative algorithms generally require intensive interprocessor communication patterns, research is conducted to determine how to optimally schedule intensive communication patterns on a proposed class of topologies called balanced generalized hypercubes (BGHCs). The standard binary hypercube topology of the nCUBE 2 is a special case of the BGHC. Performance results are derived for three intensive communication patterns as a function of a parameter of the BGHC that characterizes the sparsity of the topology.

Degree

Ph.D.

Advisors

Antonio, Purdue University.

Subject Area

Electrical engineering|Computer science

Off-Campus Purdue Users:
To access this dissertation, please log in to our
proxy server
.

Share

COinS