DEVELOPMENT AND EVALUATION OF A BEHAVIORALLY REFERENCED, MULTICOMPONENT RATING SYSTEM FOR HETEROSOCIAL SKILLS

JAN LANCE WALLANDER, Purdue University

Abstract

Recognizing the need for assessment of behavioral components of social skills, a rating instrument was developed for this purpose and psychometrically evaluated in the current research program. A modified version of the analytic assessment strategy was used in the development phase. First, behavioral samples which represented a range of competence levels were collected and displayed to potential recipients of heterosocial behavior. These recipients generated examples of appropriate and inappropriate responses within the targeted behavior class. Based on these response examples, a hierarchical component system of social skills was constructed and validated. This system provided the structure for the development of the current rating instrument. In order to avoid both narrowly defined molecular component behaviors, which may preclude generalizability and clinical application, as well as overly encompassing molar ones, which do not provide much information for treatment planning, the current components were defined at an intermediate level of specificity. Consequently, a rating methodology was considered best suited for the measurement of the components at this level of specificity. However, to attempt to compensate for any loss in inter- and intra-rater reliability resulting possibly as a function of relying on human judgment, each of the 11 components was anchored with numerous behavioral references in a manner similar to that used for BARS developed for I/O performance ratings. This was accomplished by scaling onto the rating values a large number of behavioral references which, again, were generated by potential recipients of heterosocial behavior. Thus, it was possible to directly associate most of the common acts displayed in the criterion situation with a specific value on the seven-point scales used to rate the component behaviors. This procedure resulted in the multicomponent Behaviorally Referenced Rating System for Social Skill (BRRSSS). The BRRSSS subsequently was employed in two studies in order to evaluate it utility. It was used to assess a total of 58 college-age subjects who participated in standard heterosocial interactions in the laboratory. Judges then completed the BRRSSS while watching videotape for each 30-second interval of the four-minute interactions. Based on the obtained results, it appeared that the BRRSSS has utility for the assessment of heterosocial skills displayed by college males in brief structured interactions with minimally responsive confederates. In particular, its generalizability across judges and intervals was good and it appeared not to be susceptible to inter-judge drift across periods up to seven weeks. It also evidenced good concurrent validity relationships and it discriminated among subjects who a priori were deemed likely to display different levels of social skills. These results were replicated across subject samples, including a clinical sample, and for independently trained sets of judges, but not across different assessment situations. Further, the behaviorally referenced rating format generally was found to perform better than a traditional rating format. It was discussed that this instrument, targeting intermediate level components of social skills, may prove more useful for clinical purposes than alternative approaches, although comparative data yet are lacking. At the very least, the promising results obtained thus far encourage further use and study of the BRRSSS.

Degree

Ph.D.

Subject Area

Psychotherapy

COinS