Disentangling fluency, comprehensibility and coherence: Toward a better understanding of oral proficiency profiles

Haiying Cao, Purdue University

Abstract

This study is an empirical attempt to examine how removal of all possible disfluency markers can help disentangle proficiency components assessed by the Oral English Proficiency Test (OEPT) in order to help build empirical foundations for establishing OEPT profiles. Silent and filled pauses, false starts and repetitions were removed from 50 test recordings in WAVEPAD SOUND EDITOR. Five trained OEPT raters rated the original and edited speeches. Statistical analyses addressed three research questions: 1) do fluency, comprehensibility and coherence ratings significantly change after the disfluency manipulation; 2) do correlations between fluency, comprehensibility, coherence and coherence subcomponents change after the disfluency editing; and 3) how do rater agreement and reliability change after such manipulation. The results suggest that disfluency may be a valid halo instead of a distractor: removing disfluency markers does not always lead to gain in fluency, comprehensibility or coherence scores; component correlations among fluency, coherence and comprehensibility are generally substantial while correlations among Coherence and its subcomponents are mostly very strong; in general raters' reliability was not affected by the manipulation; possible item order effects and halo effect were detected. The dissertation concludes with discussion of the results, implications for future research and limitations.

Degree

Ph.D.

Advisors

Ginther, Purdue University.

Subject Area

Linguistics|English as a Second Language

COinS