With OSEP support, researchers are investigating how data-based decision making enhances results when developing schoolwide positive behavioral support systems, using high stakes assessment scores for school improvement, and integrating curriculum-based measurement into the instructional program.
DATA INFORM SCHOOLWIDE POSITIVE BEHAVIOR SUPPORT PROGRAMS
"Data-based decision making is critical to addressing schoolwide discipline and safety issues effectively," says George Sugai. "To sustain the use of a systematic approach to positive behavioral support, procedures must be in place to enable informed and accurate decisions about whether adequate gains are being achieved and what actions should occur next."
In effective systems of behavior support, student behavior is monitored continuously, and data are used by staff as a basis for decisions. For example, Sugai and his colleagues have developed a system that uses scientifically derived information about office referrals along with other data to evaluate the effectiveness of a comprehensive schoolwide discipline and violence prevention program. Sugai's work is one of the first attempts to link a systematic analysis of office discipline referral data to inform discipline program reform efforts.
"Research suggests that office referrals provide a useful index to assess school discipline needs and monitor intervention effects," Sugai points out. "We found that school-based teams using office referral data have successfully established and maintained schoolwide discipline systems that resulted in a 50 to 60 percent reduction in rates of office discipline referrals." Sugai goes on to say that schools should use data to identify their specific needs and to determine if selected programs and interventions match their needs. When creating a data-based decision making system, Sugai suggests that educators consider the guidelines found in the sidebar, Guidelines for Creating a Data-Based Decision Making System.
Flossmoor Elementary School District 161, IL, has implemented positive behavior support programs districtwide and uses systematic data to target their efforts. "Teams in our district regularly review student data," Judith Green, assistant superintendent, tells us. "By using data, we know what to target, and how and when to intervene." Since implementing the approach, the district has seen a significant improvement in behavior.
Teachers complete referral forms when students violate discipline guidelines. A secretary enters the information into a database. Teams in each school meet monthly to review the data and make decisions. To assist their efforts, each school has access to the School-Wide Information System (SWIS) [http://www.SWIS.org], a web-based information system.
|GUIDELINES FOR CREATING A DATA-BASED DECISION MAKING SYSTEM
- Data should be readily available.
- Procedures for collecting data must be easy to use and not require excessive staff time and resources. According to Sugai, data collection systems should not consume more than one percent of someone's time each day.
- Purposes for collecting data must be relevant to ongoing activities.
- Only a small number of questions should be addressed.
"Having a database with information is helpful in focusing attention on where and when the problem is occurring," Green points out. "For example, after reviewing the data from the elementary schools, we found that many infractions were happening during recess. This enabled us to target that setting for intervention."
Statewide Positive Behavioral Support Initiative Features Data-based Decision Making
Hawaii has launched a statewide initiative to prepare all schools to use Sugai's positive behavioral support model. According to Jean Nakasato, educational specialist in the Hawaii Department of Education, a major element of the positive behavioral support system is data-based decision making.
"Our goal is for everyone to be data smart, to be able to analyze and interpret behavioral data," Nakasato tells us. "We need to know if we are meeting student needs and data help us do this."
To this end, the use of data is built into the training process. School-based teams bring data (e.g., about office referrals) to the state-sponsored training sessions where they learn how to use it. In teams, participants use the data to self-assess and problem solve. The process results in action plans that participants are expected to implement in their schools.
|TEACHING STUDENTS WITH DISABILITIES TO USE DATA FOR DECISION MAKING
"There's a saying, "In God we trust, from all else we expect data,'" asserts Tanis Bryan, researcher at the Southwest Institute for Families and Children.
To this end, Bryan and her colleagues with OSEP support developed and successfully field-tested Amazing Discoveries. In the Amazing Discoveries curriculum, students with and without disabilities in Grades 5 through 12 learn how to conduct scientific research about a topic of high interest to them-themselves! Data collection, analysis, and presentation are integral elements of the curriculum.
"Using the Amazing Discoveries approach, we have engaged youth in using data to achieve personal goals that they set for themselves," Bryan explains. "We found that once youth catch on, they quickly find other ways of using data for making decisions about even more important goals."
For more information on the Amazing Discoveries curriculum, visit the publisher's web site at www.exinn.net or contact Bryan at TanisHBr@aol.com.
USING LARGE-SCALE ASSESSMENT DATA FOR SCHOOL IMPROVEMENT
"The benefits of having students with disabilities participate in state and districtwide assessments will never be fully achieved if educators do not actually look at, explore, and use the data for making decisions about educational programs," says Martha Thurlow, director of the OSEP-funded National Center on Educational Outcomes (NCEO). "Those schools, districts, and states making the most progress are going to be the ones that thoroughly and carefully use their assessment results."
Since 1998, Thurlow and her colleagues have been studying how states and localities are implementing the IDEA requirement that students with disabilities participate in state- and districtwide assessments, with accommodations as necessary. Of particular interest is the IDEA provision requiring the public reporting of scores of students with disabilities with the same frequency as for other students.
|TECHNOLOGY STREAMLINES DATA COLLECTION AND ANALYSIS|
The latest Technology in Action issue from the Technology and Media (TAM) Division of the Council for Exceptional Children focuses on how technology can be used to manage data for schoolwide and individual student behaviors. Solving Behavior Problems: Technology Can Help! outlines suggestions for using data to determine communicative function, identifying a pattern of behavior, and analyzing and displaying group data. The issue is available on the TAM web site at www.tamcec.org.
"Reporting the scores of students with disabilities is one element of a truly inclusive accountability system" Thurlow says. "Once scores are reported, they should be used to make programmatic and instructional decisions."
To understand and use state and/or district assessment data, Thurlow suggests that educators first reflect on why it is important to look at data and how knowing such information may prove relevant for evaluating whether decision making is producing the expected results. From here, educators can decide where they might want to start their efforts and how far into the data they want to dig. According to Thurlow, this step involves knowing the data elements that are used to generate district or state reports.
"A thorough understanding of the information provided also enables us get more out of the data that are given," Thurlow adds. "It allows us to explore trends in performance, for example, changes in test scores across grades within a given year, changes in the scores of students within specific grades across years, and performance of the same students as they progress across grades. It also is important to keep track of mobility in and out of special education, and to look at data in varied ways," Thurlow says. "This is especially true when trying to reach conclusions about improvements in the performance of students with disabilities."
|DIGITAL TEXTS WITH EMBEDDED AND INDIVIDUALIZED READING STRATEGY AND DECODING SUPPORTS
Imagine a software program that presents instructional media with embedded flexible supports for learning comprehension strategies. A software program that records student data during instruction and acts on it immediately, as well as makes the data available to the teacher. Sound too good to be true? OSEP-supported researchers Bart Pisha, Bridget Dalton, and their colleagues at the Center for Applied Special Technology (CAST) have made the technology a reality, and in the coming months, it may be commercially available.
"We have been building reciprocal teaching and an array of supports into electronic copies of selected novels from a large urban school district's reading list," Pisha tells us. "The software gathers and analyzes ongoing data from the student user and scaffolds instruction accordingly." Because the software is on a network, teachers can check at any time to see how individual students are progressing. Using the data, they determine if students need additional intervention and support.
"Enhanced electronic texts can extend the capacity of teachers to support students with learning disabilities and develop the capacity of students so that they have access to, and are making progress in, the general curriculum," Pisha explains.
Here's a glimpse at how it works. The student reads the novel (or has the novel read aloud by the computer) and after every episode is prompted to execute a reciprocal strategy, such as summarize, predict, visualize, ask a question, etc. The program provides help prompts as needed according to the student's level, sometimes in the form of an animated helper. Before moving on, the student completes a work log, which is stored and analyzed. An example of a typical student's work log follows:
8:02:30 AM 12/5/02
23. Gift Giver, Level 1: Chapter Seven, Passage 1
Make a prediction about what is going to happen.
They are going to find Sherman.
Currently, Pisha and his colleagues also are working with Gallaudet University to include American sign language as an option in the program.
For more information, visit the CAST web site at www.cast.org. Or contact Pisha at email@example.com.
A District Looks at Statewide Data for School Improvement
Judy Elliott, assistant superintendent in the Long Beach Unified School District, California, has worked with Thurlow for a number of years. Her district looks closely at statewide data when making school improvement policy decisions.
"Our district's standards are aligned with the state assessment so we are very interested in our students' results," Elliott reports. Every principal in the district receives a schoolwide profile of trend lines, participation rates, and test scores. The profile shows parallel data for students with disabilities. "Data help us determine where students are making gains and where they need additional support."
According to Elliott, at first there was some concern around getting data for students who were not typically counted. IEPs were revamped to include information about the particular assessment accommodations that were being used during instruction and classroom testing. "We used this information to help us plan the best use of accommodations on the statewide assessment," Elliott explains.
To support principals in using the data, the district paid attention to information dissemination and relationship building. "We made a conscious effort to help everyone understand that the reporting requirements in IDEA and NCLB were intended to help all children achieve better results," Elliott adds.
Fact: When teachers use curriculum-based measurement (CBM)-a form of systematic progress monitoring-to track their students' progress in reading, mathematics, or spelling, they are better able to identify students in need of additional or different forms of instruction, they design stronger instructional programs, and their students make greater gains.
Fact: More than 200 empirical studies published in peer-review journals provide evidence of CBM's reliability and validity for assessing the development of competence in reading, spelling, and mathematics and document CBM's capacity to help teachers improve student outcomes.
"CBM is a powerful tool that can help teachers monitor their students' academic progress and design more effective instructional programs," says Lynn Fuchs, researcher at Vanderbilt University. "Using CBM, teachers can infuse ready-made accountability into their instructional design and can streamline the individualized education program (IEP) system to make the IEP into a living document that guides and enhances instructional decision making on a day-to-day basis."
Each CBM test assesses all the different skills covered in the annual curriculum. CBM samples the many curricular skills in such a way that each weekly test is an alternate form. Scores earned at different times during the school year can be compared to determine whether a student's competence is increasing.
"The overall CBM score can be used in three ways," Fuchs explains. "Teachers use CBM scores in universal screening to identify students in need of additional or different forms of instruction, to monitor students' development of academic competence, and to improve instructional programs." Fuchs adds that CBM skills profiles also are used to identify the skills in the annual curriculum in which students require additional instruction and to identify the students who are experiencing problems with maintaining skills after initial mastery was demonstrated.
|COMPUTER-ADMINISTERED AND SCORED CBM RESOURCES
Web-based software reduces the paper and pencil barriers to using CBM. Technology provides us with a way to efficiently analyze results, manipulate data to answer questions, and share data for decision making. Educators have several options when looking for a technology-based information management system to organize and report CBM data.
Lynn Fuchs and her colleagues also have developed software. For information, contact Fuchs at firstname.lastname@example.org.
Helping teachers use CBM also has been a focus of research for Lynn and fellow researcher Douglas Fuchs. Their work has addressed how CBM users can keep up with the mechanical tasks of measurement. "Technology can dramatically reduce the need for teachers to conduct the mechanical tasks associated with measurement, such as test administration, test scoring, graphing, and data analysis," Fuchs tells us. "Data collection software used in combination with data management software can entirely eliminate most teachers' time in such tasks." [For information on the use of technology, see the sidebar, Computer Administered and Scored CBM Resources.]
Although CBM has been commonplace in special education classrooms for years, it was not until the early 1990s that the Fuchses and their colleagues began integrating CBM into general education classrooms. "General education teachers liked CBM," Fuchs reports, "however, we needed to refocus the attention on classwide reports and skill summaries rather than individual student graphs." In addition, Fuchs found that many teachers benefited more when they received assistance in using the CBM data for decision making.
"Even though analyzing the data is helpful, in order to intervene, most teachers prefer to be given new routine ways to differentiate instruction." To help teachers break beyond their standard instructional routines and identify alternative teaching procedures, Fuchs and her colleagues have developed full-scale instructional approaches, as well as expert systems that provide instructional modifications.
District Uses CBM Data To Improve Student Achievement
We often think about CBM as a classroom intervention. But, its use school- and districtwide is growing. The Pittsfield (MA) Public Schools are a good example.
In 1997, Michael Meyers, the special education director, was introduced to Mark Shinn, University of Oregon researcher who has received OSEP support for his work in CBM over the years. Shinn's model utilizes school psychologists in key roles in collecting and analyzing the CBM data. Meyers used Shinn's work as a jumping off point to implement CBM throughout the district.
"What started as a special education initiative to identify students needing help with reading has now expanded to include every child K-3 and a plan to include all students through Grade 8," Meyers reports. CBM data are used to verify what is working and not working, and to help guide educators in selecting an appropriate curriculum. "Students are assessed every fall, winter, and spring," Meyers says. "With this monitoring, we are able to identify which students are not meeting benchmarks. With CBM we make sure every child is making progress-and if not, we intervene."
Initially, school psychologists were trained in CBM, but now, all special education teachers also are trained. One of the challenges Meyers and his colleagues faced was how to manage the data. "We found that once we went with a web-based data management program, it freed the psychologists to spend more time with teachers-helping them interpret the data and identify interventions."