New teacher evaluation systems in Tennessee have rough road ahead

by Sarah Garland

Lester School 8th grade teacher Rebecca Sellers watches on as students in her language arts class work on an assignment. Sellers has spent 17 years as an educator, including the last two at Lester. (Mike Brown/The Commercial Appeal)

Rebecca Sellers, an eighth-grade English teacher at the Lester Pre-K-8 school in Memphis, looked wary as she walked into the teachers’ lounge on a Monday afternoon last fall. The previous week, the school’s assistant principal, Isaac Robinson, had dropped in, unannounced, to watch Sellers teach as part of Tennessee’s new evaluation system.Now he was about to reveal her scores. As he fiddled with a computer connected to a projector, Robinson asked Sellers how she thought she did.
“I’m not sure how I did because I had to make some adjustments,” said Sellers. Her students tend to do well on state tests, but the lesson hadn’t gone as planned. Her eighth-graders had been stumped by a quick review exercise on pronouns. Sellers had taken an extra 15 minutes to go back over the material.
“I had to meet the children where they were at that particular time,” she added. “Do you remember the lesson?”
Robinson, a transplant from Georgia in his second year at Lester, a high-poverty school in the Binghampton neighborhood where 97 percent of students are black, muttered that he did. He had been taking notes on a new iPad provided by the school district.
With a click, he displayed Sellers’ scores on the projector: Mostly 2s, and even a 1, on a 5-point scale. Sellers, a 17-year veteran, was on track to losing tenure and possibly her job if she didn’t score at least 3s in the future.

Teacher evaluations

The Hechinger Report and Memphis Commercial Appeal recently teamed up to produce a series on new teacher effectiveness measures in Tennessee. Check back in the next several days for more stories.
You can also read our previous series on the similar issues in Milwaukee and Florida.

“Let me read what a 3 looks like: ‘Teacher communicates lesson objectives to students,’ ” said Robinson, reading from a chart on the screen. “I don’t think that was done.”
Sellers rolled her eyes. “Well, if they didn’t know what the focus was, they wouldn’t know what they were supposed to do, right? And they did what they were supposed to do,” she said.
The discussion deteriorated from there. Forty minutes later, Robinson, hunched over a keyboard, typed “off-task behavior interfered with instruction time” into a form on the computer screen. Sellers sat with her arms crossed, shaking her head.
“I don’t agree with this evaluation at all,” she said. “I don’t think it reflects the job I did.”

*

This fall, principals and assistant principals fanned out into thousands of Tennessee classrooms in an unprecedented effort to spend at least an hour annually observing and rating every single teacher, guidance counselor, social worker and librarian in the state’s public school system. Their goal: find teachers who are struggling, figure out what they are struggling with, and help them get better.
In Memphis and Shelby County, anecdotal reports suggest most feedback sessions have not been as charged as the one between Sellers and her assistant principal.

Lester School assistant principal Dr. Isaac Robinson evaluates fourth grade teacher Debra Holt-Robinson's (no relation) class at the school. Robinson observes the teacher and the lesson then moves through the class to see how well the students grasp the concepts taught. (Mike Brown/The Commercial Appeal)

“We’re not about ‘gotcha,’ ” said the Memphis superintendent of schools, Kriner Cash. “We’re not about catching teachers being level 1 or level 2 and then trying to figure out ways to get them out of the profession.”
Yet the exchange at Lester highlights the challenges both Memphis and Shelby County schools face as they roll out their new evaluation systems and attempt to retrain the local teaching force. Those challenges are why Lester’s principal, Antonio Burt, a second-year principal trained by the national nonprofit advocacy group, New Leaders for New Schools, opened the doors to a visitor.
“It’s one thing to have four or five that are rolling up their sleeves. They’re taking ownership in the work. They’re really honing their craft. But you want an entire school to be doing the same approach,” he said. “Until you get the mindset of every single individual, you won’t see that growth.”
The effort is part of a sea change in public education across the country, with Tennessee, whose students have long ranked near the bottom on national tests, at the forefront. Education reformers, including those in the Obama administration, have embraced the belief that great teaching is not an art, or, as Cash puts it, something “born in you.” Rather, they see great teaching as a science—something that can be taught and learned.
To that end, states and districts, aided by hundreds of millions of federal and philanthropic dollars, are developing intensive evaluation systems meant to identify teachers who need help, and pinpoint which skills they need help with. Under a state law passed last spring, teachers must be formally observed at least four times a year, or six if they’re new to the profession.
A teacher’s observation scores are supplemented by a so-called “value-added” rating, which is calculated by determining whether a teacher’s students made greater gains on standardized tests than statistical models would have predicted. But because value-added ratings don’t come out until after the school year is over—and because the majority of teachers don’t teach subjects with annual standardized testing—the revamped observations have become a major piece of the reform effort.
“If you look at any teacher anywhere, they all think that they’re great, and they’re all working hard and they’re trying,” said David Stephens, assistant superintendent of curriculum and instruction in the Shelby County Schools. “Sometimes they may just not have enough knowledge, or some of the skills may be lacking. So if we do some things to help improve that, then I feel like we’re headed in the right direction.”
The question is whether the new system can work where decades of other education reforms have not.
Are observations accurately identifying struggling teachers? Are teachers learning from the feedback they receive? Are they finding resources to help themselves improve? And, most importantly, are students performing better as a result?

Lester School 8th grade teacher Rebecca Sellers explains to her social studies class that learning requires more than just printing pagers but requires them to retain the information. Sellers has spent 17 years as a teacher , with the last two years at Lester. (Mike Brown/The Commercial Appeal)

“We have a need to identify our true underperformers. There are teachers that are just harmful to kids … academically harmful,” said Sandi Jacobs, vice president of the National Council on Teacher Quality, a Washington, D.C.-based advocacy group. “But we have lots of teacher who aren’t as good as they could be, and that is where the thrust of this work really is, the desire to maximize the teaching force.”
Both districts see the reforms as urgent, even though their student populations are very different. One third of Shelby County’s students are signed up for free- or reduced-price lunch, a measure of poverty, and nearly 40 percent are black, according to the state education department. In Memphis, one of the poorest cities in the nation, 87 percent of students get subsidized meals, while 84 percent are black.
Although Shelby County has one of the lowest percentages of students who are economically disadvantaged in the state, only about half of its students tested proficient or advanced on state math tests in 2011, according to state numbers. Students in both Shelby and Memphis have made big gains on math tests in recent years, but Memphis still ranks at the bottom in terms of proficiency.
In reading, gains have been smaller for both districts. In Shelby, 57 percent of students were proficient in 2011, compared to just a quarter of students in Memphis.
Halfway through the year, the Memphis and Shelby County school districts had already conducted nearly 10,000 observations of the nearly 10,500 teachers, librarians and other instructional staff in the two districts. They are already compiling data and hearing reactions—both positive and negative— from teachers and principals.

*

Many veteran teachers and principals say the biggest change this year is the amount of time principals are now spending in classrooms. Previously, teachers in Tennessee were evaluated only once every five years.
Under the new system, principals are required to spend from 60 to 90 minutes in a teacher’s classroom annually, depending on a teacher’s experience—meaning for veteran teachers, principals must conduct four 15-minute observations over the course of the school year.
In Memphis, however, district administrators have found that principals are actually spending an average of 29 minutes doing each observation, instead of the required 15. Anecdotal reports in Shelby County suggest principals there are also going beyond state requirements. In addition, conferences between principals and teachers after an observation can run as long as an hour.
Some principals and teachers have complained about the amount of time they spend doing evaluations, but others appreciate the shift. “It used to feel like I was learning things on my own. I was observed, but I didn’t get targeted feedback like I do now,” said Davida Smith-Keita, an English teacher at Manassas High School with eight years on the job. “I think it’s time well spent.”
Another Lester teacher, Brittany Edens, who began teaching this year, agreed. “The feedback has been the most valuable part, especially as a first-year teacher” she said.
Observers grade teachers on four different “domains,” including planning, teaching and classroom environment. Each domain includes a list of standards, or indicators, such as using strategies to promote higher-level thinking skills or creating a respectful classroom culture, that teachers at all grade levels, in all subjects, are expected to meet. In the classroom of an above-average teacher, for example, students should come up with their own questions and be able to explain concepts to one another with the teacher’s help. In contrast, a below-average teacher spends most of the lesson talking and only calls on volunteers, according to the standards.
During post-observation conferences, teachers receive a score on a five-point scale. Level 1 and 2 are “below expectations.” Level 3 is “meeting expectations.” And levels 4 and 5 are “above expectations.” Evaluators are supposed to point out areas in which teachers can improve and suggest how they might reach the next level.
“The substantive conversation is really the big deal. We’re already hearing from teachers that it’s a very useful experience,” said Irving Hamer, deputy superintendent of academic operations, technology and innovation for the Memphis schools.
In response to complaints about time, the state recently decided to allow observers to focus on two domains at a time during half-hour sessions, so veteran teachers will only be observed twice a year and first-year teachers will be observed three times. Memphis, which began work on overhauling its teacher evaluation program two years ago and uses a different system than the rest of the state, is deciding whether to adopt the change as well.

Some elements of the new system are similar to the old way of doing things, although the terminology has changed. During previous observations, principals took notes to create a “script” of a teacher’s lesson. Now scripting is supposed to be called “gathering evidence.” Domains with lists of indicators were part of the old system, too. But now, the number of domains and indicators has been reduced and, district administrators say, the standards have been clarified.
Before, nearly every teacher received a satisfactory score. So far, new observation scores have also skewed positive, something that neither district is celebrating.
In Memphis, more than 1,000 out of about 7,000 teachers, or about 14 percent, were rated “below expectations” in the first round of classroom observations this fall. By contrast, about 46 percent scored “below expectations” based on value-added student test scores last year. In Shelby County, evaluators have rated only about 6 percent of teachers at level 1 or 2 in their observations so far this year, compared to 25 percent who were rated level 1 or 2 based on last year’s value-added test scores.
Then there is the case of Rebecca Sellers, who received low marks during her two observations so far this year, but the top rating, 5, based on her students’ test score progress last year.
Making sure the value-added data and the observation ratings are in the same ballpark is an important test of accuracy for both measures, according to a study published last year by Measures of Effective Teaching, a research project funded by the Bill & Melinda Gates Foundation. Principals are also being evaluated by district administrators, and part of their scores depend on whether the observation ratings they give teachers correlate with the value-added data that come out at the end of the year.
If they don’t match up, the system’s usefulness and reliability could come into question, and it could lose credibility among educators.
During a recent forum hosted by TeachPlus, a national advocacy organization, about 90 teachers from the Memphis school district gathered to vent about the new evaluations over a buffet dinner of pasta and wine. A poll of the group, most of them veteran teachers, found that nearly half worried about the objectivity of their evaluators.
One teacher complained that principals were making judgments about their teaching without any context. “They don’t know the students and they don’t know what I know about the students,” she said.
But the biggest concern among the teachers was whether they would receive adequate training to get better.
In both Memphis and Shelby County, administrators say they are working furiously to improve training opportunities for teachers. Memphis is also considering some changes to the system in response to the early feedback.
Hamer, the deputy superintendent, says the district may weigh the observations differently, putting less emphasis on the first observation of the year and more on the last one so that teachers will be rewarded if they progress during the year. He said they are also looking at whether teachers should have to demonstrate everything on the list of indicators during the 15-minute observation, which is something that both teachers and principals have said may be impossible to do.
But the “major problem,” Hamer said, “is how to make sure principals are more and more effective at doing this.”

*

On a recent November morning just after announcements, Kay Obenchain, a teacher now in her seventh year, entered the principal’s office at Millington Middle School, near the northern edge of Shelby County.
About two-thirds of the students at Millington qualify for free- or reduced-price lunch, and more than a third are black. Michael Lowe, Millington’s principal for the past seven years, had observed Obenchain teach a sixth-grade math lesson earlier in the week.

Millington Middle School teacher Kay Obenchain walks through her 7th grade enriched math class to check on students as they review for testing. Obenchain has been teaching full-time at the school for 8 years. (Mike Brown/The Commercial Appeal)

Lowe pulled out a legal pad with his notes as Obenchain sat down next to him.
“How do you think the lesson went?” he asked.
“The kids seemed to enjoy it,” she said. “They learned the objective I set.”
“This question is just for reflection,” Lowe continued. “Is your room welcoming, and what evidence indicates that?”
Obenchain talked about the student work she tacked onto the walls and the grouped desks that kept students from feeling isolated. Lowe nodded and smiled. The conversation continued amicably, with Lowe praising Obenchain’s decision to hang her college degrees on the wall. He offered one tip: Obenchain should remind students of the rules before they start a lesson, which might help the giggly group in the back of the room stay on task.
At the end of the 12-minute conversation, Lowe revealed her scores: 4s and 5s. Obenchain looked relieved.
Afterwards, she praised the process: “I like it better than the old observation,” she said. “This breaks it all down so we can see what a teacher ‘above expectations’ looks like.”
Obenchain’s positive reaction to her observation is common, according to Laura Link, director of professional learning and development in Shelby County. “The ones who have been over time getting successful scores appreciate this model,” she said. In contrast, the ones who receive lower scores are more likely to complain about it.
For Rebecca Sellers, who used to teach in the Shelby County Schools but decided to return to Memphis because she believed her skills were needed in the district’s high-poverty schools, the experience has led her to consider leaving the profession. “All of a sudden this year I’m not doing it right,” she said. “The joy of teaching is almost gone.”
It’s likely that both sets of teachers, those identified as high- and low-performers, will need to buy into the model for it to translate into better achievement for students, since administrators in both districts say they plan to replace only a handful of teachers—those at the very bottom of the ratings.
“It’s imperative for the district office to give schools and teachers the tools that they need,” said Stephens, the Shelby assistant superintendent. “This evaluation thing is not a ‘gotcha.’ It’s how can we help you improve.”
A version of this story appeared in the Memphis Commercial Appeal on February 5, 2012.