Evaluating Teachers Using Student Test Scores: IMPACT (Part 2) by larrycuban

Since 2009, a new system of teacher evaluation has been put into practice i nWashington, D.C. called IMPACT. The “Teaching and Learning Framework” for IMPACT lays out a crisp definition of “good” teaching in what D.C. teachers call the “nine commandments”: 1. Lead well-organized, objective-driven lessons. 2. Explain content clearly. 3. Engage students at all learning levels in rigorous work. 4. Provide students with multiple ways to engage with content. 5. Check for student understanding. 6. Respond to student misunderstandings. 7. Develop higher-level understanding through effective questioning. 8. Maximize instructional time. 9. Build a supportive, learning-focused classroom community. IMPACT uses multiple measures to judge the quality of teaching. Fifty percent of an annual evaluation is based upon student test scores; 35 percent based on judgments of instructional expertise drawn from five classroom observations by the principal and “master educators,” and 15 percent based on other measures. Using these multiple measures, IMPACT has awarded 600 teachers (out of 4,000) bonuses ranging from $3000 to $25,000 and fired nearly 300 teachers judged as “ineffective” in its initial years of full operation. For those teachers with insufficient student test data, different performance measures were used. Here is a description of one classroom being observed by a “master educator.” "A case in point is the lively classroom of Andrea Stephens (not her real name), a first-grade teacher at a racially mixed elementary school in Northeast D.C. Master educator [Cynthia] Robinson-Rivers is conducting an informal observation as Stephens teaches a lesson about capital letters, punctuation marks, and the short “a.” Stephens is kind, firm, and engaging, and she wins points for gestures like asking a reluctant pupil if she could “get one of his smiles,” making him feel valued. But she is apparently not engaging enough. Several students are not paying attention; one is a mugger and a performer, and he can’t sit still. After several attempts to quiet him, Stephens gently pulls him up next to her, holding his hand while she addresses the rest of the class. The general atmosphere suggests to Robinson-Rivers a need for better management. “The children weren’t completely out of control,” Robinson-Rivers says. “But if they  aren’t facing you it can suggest a lack of interest.” The session reveals other perceived shortcomings, despite Robinson-Rivers’ respect for Stephens as “a warm, thoughtful practitioner.” It was too teacher directed, Robinson-Rivers says; it failed to make the objectives fully clear, and it didn’t make the most of limited instructional time. “If the pacing is too slow, you can lose valuable time from the lesson,” Robinson-Rivers says. “If in a 20-minute morning meeting the kids participate in a variety of engaging activities, it’s much easier to maintain their interest and enthusiasm.” Stephens also falls short on Teach 5 [one of the “commandments”]—checking to see whether students actually understood her. “There was no way to know whether the shy girl or the boy who spoke little English understood or not,” Robinson-Rivers says. Instead of having all the pupils answer in unison, she suggests that Stephens cold-call on individual students, or have all the boys or all the girls answer in some non-verbal way. “It’s hard because teachers do think they are checking for understanding. But it’s actually an easy one for professional development; you could just say there are three easy things you can do.” Stephens, whose overall score for the year was in the “effective” range, is open to evaluation and receptive to feedback—she even asked for an extra observation—and in this regard, master educators say she is fairly typical." Just as any system of evaluation with observers using subjective criteria can distinguish between acceptable and unacceptable performances, IMPACT does sort out on a four-point scale “effective” from “ineffective” teachers according to their nine commandments of “good” teaching. That the use of VAM and other indicators reward and punish teachers is also clear. Questions, however, about teacher turnover—has IMPACT reduced attrition among “effective” teachers?—and student performance—has IMPACT improved test scores? have gone unanswered thus far. Allegations of teachers and administrators erasing student answers and fiddling with tests further cloud the picture of IMPACT’sinfluence on teaching practice and student achievement. Among teachers and principals, the degree to which IMPACT has influenced daily teaching is disputed. According to some teachers, there are colleagues who pull out special lessons when principals and “master educators” appear for 30-minute unannounced visits. Other teachers tremble and panic when an evaluator walks into their classroom and the lesson becomes a shambles.  And there are many teachers who relish the feedback they get from conferences after observations and assert that they have made changes and their lessons have improved. While the principal’s workload demanded by teacher observations has become unmanageable for some, others have welcomed the role of instructional leader. Some principals, however, find IMPACT destroying their ways of supervising  teaching in their schools. What has occurred in Washington, D.C. with new curriculum standards, new tests, and IMPACT mirrors what has occurred in urban districts across the country that have put into place testing and accountability structures. Since the early 1990s and especially after the passage of No Child Left Behind most urban schools across the nation have narrowed their curriculum leaving less time for non-tested subjects, intensified teacher-centered instruction with more lessons devoted to preparing students for tests, and, in general, reduced instructional time for reaching all of the curricular standards they are expected to meet over the school year. New ways of evaluating teachers that anchor judgments of effectiveness, in part, on student test scores certainly reinforces accountability but to what degree teaching practices across elementary and secondary school classrooms have improved as a consequence of IMPACT or similar programs is yet undetermined. And whether those improved teaching practices have led to gains in student academic achievement, well, no one knows.