26 de fevereiro de 2012

In Teacher Ratings, Good Test Scores Are Sometimes Not Good Enough



Joshua Bright for The New York Times
At Public School 87 on the Upper West Side, just over half the ratings were above average.



At Public School 234 in TriBeCa, where children routinely alight for school from luxury cars, roughly one-third of the teachers’ ratings were above average, one-third average and one-third below average.
Multimedia
Metro Twitter Logo.
Follow @NYTMetro
Connect with us on Twitter for breaking news and headlines in New York.
Hiroko Masuike/The New York Times
At P.S. 234 in TriBeCa, the ratings were split almost evenly among above average, average and below average. The ratings were released Friday.
At Public School 87 on the Upper West Side, where waiting lists for kindergarten spots stretch to stomach-turning lengths, just over half the ratings were above average. The other half were average or below average on measure, based on student test scores.
And at Public School 29 in Cobble Hill, Brooklyn, where many young parents move to send their children, four of the teachers’ ratings were above average, seven were average and five were below.
The New York City Education Department on Fridayreleased the ratings of some 18,000 teachers in elementary and middle schools based on how much they helped their students succeed on standardized tests. The ratings have high margins of error, are now nearly two years out of date and are based on tests that the state has acknowledged became too predictable and easy to pass over time.
But even with those caveats, the scores still provide the first glimpse to the public of what is going on within individual classrooms in schools. And one of the most striking findings is how much variation there can be even within what are widely considered the city’s best schools, the ones that each September face a crush of eager parents.
At Public School 321 in Park Slope, Brooklyn, for example, 10 teacher ratings were above average, 13 were average and 5 were below. At Public School 89 in TriBeCa, one of six rated math teachers received higher-than-average rankings, a lower rate than in the city as a whole. In many cases, teachers received two career ratings, one for math and one for English.
The principal cause of the wide variation within schools is the methodology of the ratings, which compares teachers with similar student demographics and scores. For teachers in schools with high-achieving students, good test results are often not good enough, at least by the standards set by the formula.
The city acknowledged that the model was “too sensitive” among teachers whose students did either very well or very poorly. That lesson was shared with schools and the state “as it creates a new model for teacher evaluations,” said Matthew Mittenthal, an Education Department spokesman.
The data, for example, showed 73 cases in which teachers whose students produced consistently outstanding test scores — at or above the 84th percentile citywide — were nonetheless tagged as below average. The reason? The formula expected even better results, based on the demographics and past performance of the students.
While the formula is only one aspect of teacher evaluations, the system represents a sea change in that teachers of such high-performing students will be held accountable for student improvement even if their students are far from failing.
In one extreme case, the formula assigned an eighth-grade math teacher at the prestigious Anderson School on the Upper West Side the lowest possible rating, a zero, even though her students posted test scores 1.22 standard deviations above the mean — normally good enough to rank in the 89th percentile. Her problem? The formula expected her high-achieving students to be 1.84 standard deviations higher than the average — roughly the 97th percentile.
Not all teachers who work with the city’s better-off students can be “above average,” said Sean Corcoran, an associate professor of educational economics at New York University who has written about the city’s scores, developed through a system called value-added modeling.
“The value-added measures are controlling for many of these factors,” Mr. Corcoran said, “so even if you just take the top students in the city, there is going to be a full distribution of teachers working with them, from high scoring to low.”
For parents, seeing the rankings of the teachers they know well can be shocking. Vicki Kahn, a parent at Public School 333, the Manhattan School for Children on the Upper West Side, was surprised to see that some of the teachers whom she considers outstanding had poor ratings, including one who routinely sends many of her students to a highly selective middle school.
“It seems completely wrong,” Ms. Kahn said, adding that one of the co-teachers in the sixth grade did not even get a rating, in an apparent mistake.
Anna Rachmansky, whose son is a fifth grader at P.S. 89 in TriBeCa, was visibly stunned upon discovering that a teacher she held in high regard scored in the 10th percentile in math.
“I’m very surprised she would get poor in anything,” Ms. Rachmansky said. “She’s a very strong educator.”
Sandra Blackwood, the co-president of the parent teacher association at Public School 41 in Greenwich Village, said she had little confidence that the data would be meaningful, though she felt it was important to look.
“If it is anything like the school grading system,” she said, “it will probably be highly arbitrary.”
Though parents can get a peek inside school buildings for the first time to see differences among teachers, it does not help if the underlying information is incorrect, Elizabeth Phillips, the principal of P.S. 321, said.
“What people don’t understand is that they are just not accurate,” she said. “We are talking about minute differences in test scores that cause a teacher to score in the lowest percentiles,” like a teacher whom she finds great and who scored in the sixth percentile because her students went to a 3.92 average test score from a 3.97, out of a possible 4.
And that is not to mention one teacher who had test scores listed for a year she was out on child care leave, Ms. Phillips said.
“The only way this will have any kind of a positive impact,” she said, “would be if people see how ridiculous this is and it gives New York State pause about how they are going about teacher evaluation.”
Despite the problems in the data, Dr. Corcoran said he did think the rankings could be useful in conjunction with classroom observations and what a parent already knew about a teacher. “Even if you don’t learn anything about teachers in the middle, which is most of them,” he said, “maybe you can see who the standouts are and who are the lowest performers.”

Alex Vadukul contributed reporting.

Nenhum comentário:

Postar um comentário