On mutation and data flow

Weichen Eric Wong, Purdue University

Abstract

Several researchers have reported results based on comparisons among data flow adequacy criteria. However, there is a lack of such comparisons between mutation and data flow. In the first part of this dissertation, we report experiments to compare the relative strength, cost, and fault detection effectiveness of mutation, its variants, and data flow based adequacy criteria. The results of these experiments can benefit a tester in selecting an adequacy criterion to construct and/or evaluate a test set under different time, budget, and criticality constraints. In spite of the empirically demonstrated superiority of mutation over the path-oriented testing strategies in fault detection, it is not used in practice primarily because of its high operational cost. In the second part of this dissertation, we propose alternative mechanisms to improve the performance of mutation testing. Results from an empirical study to evaluate these mechanisms favor the use of mutation, for example, to test safety-critical parts of a software. It is often impossible to run all test cases generated to satisfy a software's release criterion during regression testing. This inspired us to explore how regression testing can be conducted economically and effectively. In the third part of this dissertation, we examine the effect of test set minimization on the fault detection capability of data flow criteria. An answer to this question provides guidelines to a tester on how to select a representative subset of test cases from a test suite. In the last part of this dissertation, we report on experiments that reveal that test cases selected based on a coverage criterion are more likely to reveal faults in programs than those generated randomly. These results support the use of partition testing as opposed to random testing for building and evaluating test sets.

Degree

Ph.D.

Advisors

Mathur, Purdue University.

Subject Area

Computer science

COinS