Development and validation of the Roadrunner Behavioral Rating Scales for identifying participants for the Future Problem Solving Program
Abstract
The primary objective of this study was the production of a practical assessment tool that teachers in the Future Problem Solving Program could depend on to give them appropriate information about their students' capabilities. The instrument is known as the Roadrunner Behavioral Rating Scales (Roadrunner). The initial version of the instrument was developed and field tested during a series of preliminary studies. For the present study, a revision of the initial version was completed and field tested for reliability, validity, and practical utility. The performance of the current version of the Roadrunner scales was tested under regular classroom conditions. Two teams of judges produced team observation scores for 120 children. Analyses of the data showed strong interrater reliability estimates. Estimates of internal consistency reliability and test-retest reliability were very strong. Factor analysis evidence demonstrated independence of scales: Academic Characteristics, Written and Oral Communication, Creative Behavior, Interpersonal Skills, Personal Characteristics. Statistically significant levels of moderate to strong congruent validity were established with selected scales and subtests of two instruments, the Scales for Rating the Behavioral Characteristics of Superior Students (Renzulli and Hartman, 1971; Renzulli, Smith, and others, 1976) and the Stanford Achievement Test (8th ed.) (Psychological Corporation, 1989a, 1990). The Roadrunner scales operated appropriately to discriminate between identified gifted and non-identified gifted students. Content validity was established through expert review, as was the practicality of the instrument format. The Roadrunner Behavioral Rating Scales produced strong evidence of its psychometric properties. Reliability was established based on evidence from studies of interrater reliability, internal consistency reliability, and stability reliability. Confirmation of construct validity in its broad inclusive conception (Gronlund and Linn, 1992) was produced in a study of content validity through expert review, studies of congruent validity with other instruments, and studies of the instrument's power to discriminate between relevant criterion groups. Factor analysis confirmed the independence of the scales. FPSP experts and classroom teachers approved the practical utility of the instrument. The Roadrunner scales were demonstrated to be a useful tool for teachers to use in identifying students for the Future Problem Solving Program.
Degree
Ph.D.
Advisors
Linden, Purdue University.
Subject Area
Educational psychology|Psychological tests|Educational evaluation
Off-Campus Purdue Users:
To access this dissertation, please log in to our
proxy server.