Capturing L2 Oral Proficiency with Caf Measures as Predictors of the Actfl Opi Rating

Mayu Miyamoto, Purdue University

Abstract

Despite an emphasis on oral communication in most foreign language classrooms, the resourceintensive nature (i.e. time and manpower) of speaking tests hinder regular oral assessments. A possible solution is the development of a (semi-) automated scoring system. When it is used in conjunction with human raters, the consistency of computers can complement human raters’ comprehensive judgments and increase efficiency in scoring (e.g., Enright & Quinlan, 2010). In search of objective and quantifiable variables that are strongly correlated with overall oral proficiency, a number of studies have reported that some utterance fluency variables (e.g., speech rate and mean length of run) might be strong predictors for L2 learners’ speaking ability (e.g., Ginther et al., 2010; Hirotani et al., 2017). However, these findings are difficult to generalize due to small sample sizes, narrow ranges of proficiency levels, and/or a lack of data from languages other than English. The current study analyzed spontaneous speech samples collected from 170 Japanese learners at a wide range of proficiency levels determined by a well-established speaking test, the American Council on the Teaching of Foreign Languages’ (ACTFL) Oral Proficiency Interview (OPI). Prior to analysis, 48 Complexity, Accuracy, Fluency (CAF) measures (with a focus on fluency variables) were calculated from the speech samples. First, the study examined the relationships among the CAF measures and learner oral proficiency assessed by the ACTFL OPI. Then, using an empirically-based approach, a feasibility of using a composite measure to predict L2 oral proficiency was investigated. The results revealed that Speech Speed and Complexity variables demonstrated strong correlation to the OPI levels, and moderately strong correlations were found for the variables in the following categories: Speech Quantity, Pause, Pause Location (i.e., SILENT PAUSE RATIO WITHIN AS-UNIT), Dysfluency (i.e., REPEAT RATIO), and Accuracy. Then, a series of multiple regression analyses revealed that a combination of five CAF measures (i.e., EFFECTIVE ARTICULATION RATE, SILENT PAUSE RATIO, REPEAT RATIO, SYNTACTIC COMPLEXITY, and ERROR-FREE AS-UNIT RATIO) can predict 72.3% of the variance of the OPI levels. This regression model includes variables that correspond to Skehan’s (2009) proposed three categories of fluency (speed, breakdown, and repair) and variables that represent CAF, supporting the literature (e.g., Larsen-Freeman, 1978, Skehan, 1996).

Degree

Ph.D.

Advisors

Fukada, Purdue University.

Subject Area

Linguistics|Bilingual education|Communication|Computer science|Education|Foreign language education

COinS