| 1999 Study Shows Students Learn Assertiveness From Software |
|
||
COMPUTER BASED DELIVERY OF PREVENTION TRAINING A PILOT EVALUATION STUDY Ripple Effects for Teens is a groundbreaking, computer-based, comprehensive prevention program, covering health, safety and discipline issues, including violence, substance abuse, sexuality , abusive relationships, discipline and other issues. It targets a broad range of adolescents, in a variety of school and community-based settings. Ripple Effects for Teens integrates research from education, psychology and prevention studies, with advances in technology, to make best practices in social learning available to adolescents and the practitioners who work with them. The program is built around ripple effects proprietary "whole spectrum" learning system, which offers experiential, cognitive, behavioral and affective learning strategies, as well as transfer training opportunities with friends and family, media analysis and sports applications, role plays, assisted writing exercises to help internalize learning, subjective assessment, objective assessment through engaging interactive exercises, and a management system for tracking student skill mastery while protecting student privacy. The goals of the program are to develop social competence in adolescents and to help them use specific social emotional abilities to solve everyday life problems. The seven core social-emotional competencies are: empathy, assertiveness, impulse control, management of feelings, decision-making ability, self understanding and connection to community. They are broken down into 100 skill components that are presented in an array of multimedia forms that appeal to a wide range of learning styles, intelligences, social backgrounds and special needs. THE RESEARCH QUESTION The most important research question regarding the Ripple Effects for Teens software for use as a prevention program is not whether the combination of strategies used in the program can be effective in developing social competence and reducing involvement with drugs, alcohol, violence, early sexual activity, and unsafe social and personal behavior. Each of the strategies has already been effective in other studies. Rather it is whether locating the content expertise within the computer, instead of with a live instructor, will still allow students to learn effectively. (All other factors being equal, such as sufficient role play, practice opportunities and transfer training.) Previous studies had shown that academic learning could happen successfully via computer, the question is whether social learning could also happen effectively via computer technology. OVERVIEW The study, described more fully below, showed that even very short term, computer based delivery of at least some model program components, using multi-media, interactive exercises, and an engaging hip hop style that students like, results in knowledge gains that are statistically significant and compare favorably with those that are the result of longer term programs delivered by live instructors. RESEARCH METHODS The first evaluation of Ripple Effects for Teens focused on content mastery (knowledge gains), using traditional research methods, including pre and post testing and a control group matched to the experimental group. CONTENT AREAS The pilot study focused on one skill that is positively correlated with reduced involvement in drugs and alcohol, and negatively correlated with involvement in violence and exploitation, from both victim and perpetrator perspectives: assertiveness. Specifically, it covered general assertiveness skills, including assertive use of voice, eyes, posture, message, as well as setting limits, expressing feelings, giving compliments and resisting pressure, all social skills connected to reduced involvement in drugs and alcohol, reduced pregnancy and involvement in unsafe sex, and reduced chance of being a victim - or perpetrator - of violence or abuse. INCLUSION OF FEMALES AND MINORITIES The pilot study was comprised of two mixed gender groups, with 20 participants in the experimental group and 21 in the control group. Both groups were racially diverse, including Black, Latin, Asian and Caucasian participants. Percentages of each group differed from the population as a whole in two ways: there was less representation of Caucasian participants than in the general population (typical of most urban, public schools), and there was more representation of Asian students than any other group (typical of San Francisco public schools). Each group was given a pre and post test measuring assertiveness level. COMPUTER TO STUDENT RATIO Although the original intent in the pilot study was to have no more than two students per computer, technical complication in the computer lab resulted in just six usable computers being available for the study. The 20 students in the experimental group self selected partners to share computers. Groups ranged from 2-6 students per computer. This grouping of students inadvertently turned out to be positive, with much learning taking place through peer interaction. For this reason, shared computers that force peer interaction will be recommended as the norm in program implementation. A new question now is whether similar learning could take place in a private space, without peer collaboration. To answer that, at least one future test will be in private settings with a single individual on a computer. DESCRIPTION OF INTERVENTION Using no instructor supplied content, only computer based activities and peer reflection and collaboration, students in the pilot experimental group were assigned the tasks of reviewing the "info" and "how to" screens, then completing the journal writing activities and interactive assessment exercises for each of five topic areas: assertiveness, setting limits, expressing feelings, giving compliments and resisting pressure. The facilitator simply directed learners in use of the program and encouraged them to follow their own learning style in how to pursue it, but required completion of interactive activities. She also answered technical questions and/or responded to computer failure, kept order, made sure they traded off computer "driving" time and checked the computer records to be sure the interactive exercises had been completed by each group. All of this was done in a single class period. INTERNAL CONTROLS FOR VALIDITY In the pilot, there was no attrition, due to the intervention taking place in a single class period. Pre and post tests were given to the same students on the same day for the experimental group, and to the same students two days apart in the control group. (Control group students could not be convinced to take the same test twice within less than an hour.) The pre and post survey administrator was an outside person previously trained to administer the CDC's Adolescent Risk Behavior Survey. RELIABLE AND VALID OUTCOME MEASURES To maximize the validity of the pilot test, no new instrument was developed for the evaluation. Rather an existing instrument, already confirmed as reliable and valid, was used: Children's Assertiveness Behavior Scale (CABS).2 In previous studies the CABS test was shown to have a test-retest reliability rating over a four week period of .87. CABS scores correlated with both teacher ratings and behavioral observations, with behavioral observation correlated 0.38 with the CABS scores. Teacher ratings showed significant but highly variable correlations. CABS has been shown to posses discriminative validity and could accurately differentiate students who received 16 versus 8 hours of skill training in previous studies. The intervention studied here was only one hour, and used only 1/3 of the items measured on the CABS test. ANALYSIS APPROPRIATE TO DATA The group of students who participated in the pilot Ripple Effects for Teens computer based training showed a statistically significant reduction in aggressiveness as measured by AGCABS (the aggressiveness subscale), from 3.55 to 2.10 (on an 18-point scale; p=0.015) and on the overall CABS scale (which treats passivity and aggressiveness as equally problematic) from 6.15 to 4.05 (also on an 18-point scale; p=0.014). The passivity index (PACABS) showed a trend towards lower levels of passivity (from 2.60 to 2.95) which was not statistically significant (p=0.449). There was no significant test-retest effect in the control group. PACABS moved slightly, from 2.58 on the first survey to 2.63 on the second (p=0.774); AGCABS changed from 3.21 to 2.95 (p=0.368); CABS changed from 6.06 to 5.88 (p=0.587). Means were reported for ease of interpretation. However, because of the discrete scale of the outcome measures, the non-parametric Wilcoxon Signed-Ranks test was used to determine statistical significance. This test relies on relative ranks rather than on a direct comparison of means, since the non-normal distribution of scores violates the assumptions of the t-test for equality of means. For the treatment group, there were significant gender differences both pre- and post-test in passivity, but not in aggressiveness or assertiveness. In the control group the difference for CABS was also significant pre and post. There were no significant differences between genders in the overall effectiveness of the treatment. Data was not aggregated or disaggregated by race or ethnicity, partly to streamline the approval process. SIGNIFICANCE OF FINDINGS The pilot study involved a small sample over a short term. Nonetheless, it has landmark significance. If results can be expanded with further testing, it will mean that "experts" will no longer be required to deliver prevention programming to young people. Community based organizations will be able to hire ever greater numbers of natural leaders, who may lack education credentials but elicit respect. Classroom teachers who have been buckling under the burden of communicating content they have no training in, will be relieved of that burden and yet will have effective tools for promoting prevention of a wide range of unsafe and unhealthy behavior, as well as promoting social-emotional learning and dealing with discipline issues among their students. POTENTIAL FOR INTERNET DISTRIBUTION If similar results can be gained with internet delivery of the product, it will mean that the growing number of organizations who have internet access, but not CD ROM machines, will have access to effective, prevention training. Perhaps more importantly, adolescents, especially those already alienated from institutions will be able to get training in effective prevention methods at home, with their friends - or alone - when they are most motivated to do so. ESSENTIAL CONDITIONS NECESSARY TO REPLICATE IT WITH FIDELITY The first condition essential to replicate the program with fidelity is access to adequate computer technology. The program requires a Pentium 1 or later processor using Windows 95 or later system; or a Mac Quadra 640 or later, with 12 MB of processing memory, a 4x CD-ROM drive, a sound card and at least 400 MB of storage memory. Headphones are recommended so students can have privacy in exploring sensitive topics. Theoretically the program can run with only 32 MB of storage memory, but that requires switching disks, not a realistic proposition in most settings. (7a) The second condition is a structured setting, with supervision. Like other serious tools, this program cannot be set out on a table and expected to work on its own. It needs to be placed into a context where it is being used toward a specific end. The third condition is to require, at the least, completion of the interactive exercises (brain/journal and assessment exercises) for whatever topic is at hand. These learn-by-doing experiences are critically important for the many at-risk kids who are experimental learners and simply don't learn from reading. EVALUATION FEATURES IMPORTANT FOR REPLICATION While not absolutely essential to use of the program, peer collaboration is a very important factor in its success. It may be tempting for facilitators to over-control the presentation setting and substitute their responses and guidance for that of other teens. This would be a mistake. Necessary correction for wrong answers will come from within the program, without the embarrassment that often accompanies adult correction. Completion of the "passive" reading screens does not appear to be necessary for learning for many teens, and need not be required in replicating the program. TRAINING AND SUPPORT AVAILABLE There is extensive support for those seeking to replicate the Ripple Effects for Teens program:
PLANNED FOLLOW UP STUDIES Follow up studies will include replication of this study with a larger sample size, additional studies of knowledge gain in six other core social-emotional competencies, development of new measurement instruments, and longer-term, outcome-oriented studies. Ripple Effects encourages users to collect pre and post outcome data. To motivate that, the Company offers discounts for participating in research, and GUARANTEES positive improvements along no fewer than three of 28 parameters having to do with Safe and Disciplined schools, or users can receive their MONEY BACK. PARAMETERS INCLUDED IN GUARANTEE INCLUDE Decreases in Increases In CONTACT INFORMATION TO PARTICIPATE IN FUTURE STUDIES To participate in future research on Ripple Effects social learning software applications, please contact: Robin Stern, Ph. D. or Alice Ray, President & CEO
|
|||
Copyright 1998-2007 | Contact us | 1-888-259-6618 | Privacy |
