Core principles and test item development for advanced high school and introductory university level food science

Claudine A. M Laing-Kean, Purdue University

Abstract

Programs supported by the Carl D. Perkins Act of 2006 are required to operate under the state or national content standards, and are expected to carry out evaluation procedures that address accountability. The Indiana high school course, Advanced Life Science: Foods (ALS: Foods) operates under the auspices of the Perkins Act. However, no broad based course assessments were in place to measure enrollees’ performance, creating a need for standards-based assessments for ALS: Foods. The following questions were posed for this research: (1) What are the core principles and respective performance indicators for advanced high school/introductory university-level food science? (2) To what extent do potential test items generated from the performance indicators represent each core principle and adequately sample the full content area? (3) What core principles are most important to the advanced high school/introductory university-level food science curriculum? (4) What are the initial statistical verifications for these test items? This preliminary investigation involved three phases. Phase I employed a Modified Delphi Method. Panel of expert from a wide variety of food science fields were the central instrument used in this research framework. This panel (the Modified Delphi Panel) of food science professionals completed five rounds of online surveys through which they (1) reached agreement on the researcher’s proposed core principles for advanced high school/introductory university-level food science and (2) arranged and refined performance indicators for the core principles, and rated them as Essential, Important, or Optional. In Phase II, for the development of a pool of potential standards-based test items, the researcher used the published version of the performance indicators for ALS: Foods and not the version modified by the Delphi Expert Panel in Phase 1, because the high school instructors who participated were not aware of the changes in the wording of the existing performance indicators, and to some extent the content of the new performance indicators. These test items were screen tested with students enrolled in the ALS: Foods course (n = 123). Each participating teacher received a tailor-made test with test items for the performance indicators he or she had specified. The data from Phase II informed the researcher about the problems that might occur during the pilot-testing process, and the probable performance of the test items for the target population. In Phase III, a second expert panel, the Test Item Review Panel [food science professionals: from nationwide food industry (n = 4), instructors from higher educational institutions situated throughout the United States (n = 5), and instructors from Indiana high school settings (n = 6)] performed content-related validation on the test items and made test specification recommendations via six online surveys. Also, enrollees in ALS: Foods (n = 132) participated in pilot testing of the items and comprehensive standards-based test forms. Through these processes, ten core principles (Chemical properties of foods, Physical and chemical interactions in food, Nutritional composition of foods, Food laws and regulations, Food safety, Uses of microbes in food production, Sensory analysis, Food processing and preservation, Food packaging, Biotechnology and genetically modified food) with 85 corresponding performance indicators were identified (47 of the performance indicators were classified as Essential to advanced high school/introductory food science curricula, 26 were Important, and 12 were Optional), and 158 standards-based test items from a pool 184 were approved. These test items were provided to the research funder and remain confidential. A Table of Test Specifications for a comprehensive standards-based ALS: Foods assessment was created based on the rating of the performance indicators (in Phase I), classification of the test items and the distribution of the performance indicators' topics (in Phase III). The Test Item Review Panel recommended that a comprehensive, end-of-course assessment be proportioned to include 65% of topics/performance indicators deemed Essential, 25% Important, and 10% Optional. For the pilot testing, the statistical analyses indicated that 54 (34.2%) of the test items had moderate to strong discrimination values that ranged from .30 to .40, while 77 (48.7%) of the test items had moderate to strong difficulty values of .30 ≤ p ≥. 70 based on the test-takers performance in this research. The main medium for the operations of the research framework was through content area experts' viewpoints. This research demonstrated that the use of content area experts' viewpoints in making decisions about curricula and assessments can be effective, especially through the Modified Delphi method, and online survey research methodology. This methodology enabled the experts to make informative decisions to establish content validity of test items, and contributes to alignment of education, business, and industry expectations. Additionally, item discrimination, item difficulty, and distractor efficiency analyses provided quantitative data to develop high quality test items. (Abstract shortened by UMI.)

Degree

Ph.D.

Advisors

Fox, Purdue University.

Subject Area

Home economics education|Educational tests & measurements|Curriculum development|Food Science

COinS