March 7, 2011
Premium article access courtesy of Edweek.org.
Detroit
Standardized test score gains in 304 schools in six states and the District of Columbia were so improbable they should be investigated, an analysis by a group of newspapers has found.The review used open-records laws to obtain the data, then compared year-to-year changes in students' test scores and singled-out grades within schools for which the gains were three standard deviations or more from the average statewide gain on a test.
Overall, the analysis questioned gains in Arizona, California, Colorado, Florida, Michigan, Ohio, and the District of Columbia.
The Detroit Free Press, as part of the investigation with USA Today, the Arizona Republic, and other publications, looked through millions of test score results. Its review found 34 schools in Michigan—all but two of them in the Detroit area—with test score deviations over a one-year period, 2008-2009, that statistical experts said were virtually impossible to explain through improved instruction alone.
The Michigan Department of Education said it would look into the findings. "The state is committed to high-quality assessments and accountability," said Jan Ellis, a department spokeswoman.
So much is riding on test results in the current environment that their integrity and accuracy are more important than ever, experts say.
In a number of states, teacher pay already is linked to test scores. Several school districts in Michigan have tied teacher salaries to test scores, and under a 2009 state law, districts must develop plans to make student test scores a significant part of teacher evaluations. Michigan also plans to raise the bar for what is considered proficient on the MEAP test, saying schools aren't adequately preparing students for college.
The Arizona Republic also found big jumps in test scores in 93 Arizona schools. It found average scores in at least one grade in each school took a one-year jump that testing experts say is statistically improbable. Experts told the newspapers that at least one group of students at 22 of the schools failed to achieve similar scores the next year. That's a sign that the prior year's tests merited more investigation, according to experts interviewed by the papers. The odds of those jumps occurring by chance ranged anywhere from 1 in 370 to 1 in 396 million.
Arizona Attorney General Tom Horne and current state superintendent John Huppenthal said they don't believe that test-giving improprieties are widespread. Horne was schools chief during the five years reviewed by The Republic.
The anomalies, found using a methodology widely recognized by mathematicians and testing experts, are not proof that any cheating on AIMS by educators occurred at those schools. The Republic contacted officials at more than a dozen schools with especially large gains, and all insisted the increases were real.
Stepped-up scrutiny of tests is more imperative than ever because the incentive for educators to manipulate scores is only growing, experts say. Low scores on achievement tests increasingly can affect teachers' performance ratings and pay and can lower a school's ratings, leading to state intervention and loss of students to competitors.
"There's pressure from the superintendent and pressure from the state, and so the pressure causes some people to cave in," said Tom Haladyna, an emeritus professor at Arizona State University and a nationally known researcher on standardized testing.
Arizona currently does not screen for suspicious swings in scores as it watched out for the integrity of its Arizona's Instrument to Measure Standards test. It focuses instead on prevention by training the test givers, and it responds to complaints. In fall 2009, the state did start checking erasure marks on AIMS tests.
The data analysis used by The Republic is similar to one used by the Florida Department of Education to identify schools with unusual gains. The Republic examined AIMS scores in Grades 3-8 at 1,337 schools over five years. It looked for groups of students, an entire grade, where the average score jumped by at least three standard deviations. Standard deviation measures variation from the average, and testing experts say a jump of three or more standard deviations is statistically improbable. The Republic found 123 instances of such jumps at 93 schools during the five years.
The schools are of various types, including neighborhood schools in large districts, schools that serve mainly poor families, and charter schools.
Principals and teachers say their schools' extraordinary gains on AIMS are real and the result of their focus on student improvement. They also attributed gains to skilled teachers, changes in curriculum and hard work. Score drops are often caused by students moving to less experienced teachers and students moving to different schools, they said.
Consider Friendly House Academia del Pueblo, a charter school in central Phoenix. In spring 2009, mean AIMS math scores in about 25 students in the seventh grade shot from near the bottom to near the top of all of the state's seventh-graders. The following year, they plummeted.
Principal Ximena Doyle attributed the subsequent drop to the students having a different teacher that year.
Another school with remarkable gains was Kenilworth Elementary School in central Phoenix. A strong team of seventh-grade teachers, an especially good math teacher and motivated students were responsible for significant gains in two grades, Principal Ed Flores said.
Carpe Diem Collegiate High School and Middle School in Yuma had some of the largest gains on AIMS in the state. School officials credited increases to the school's unique learning model that combines classroom, online and video instruction, along with frequent testing to assess where students need to improve.
Huppenthal said he is not pushing to overhaul the way schools or state officials investigate or detect cheating on tests. The vast majority of schools operate with integrity, he said.
Still, "you're fooling yourself if you don't think there are issues out there," he said. "So we are going to be alert and examine those issues. But I think the system they have in place has been a system that works, with a few exceptions."
Nenhum comentário:
Postar um comentário