Pay-for-performance for school students is no silver bullet

Satchel, uniform, bonus


May 20th 2010 | From The Economist print edition

POLITICIANS around the world love to promise better education systems. Proposals for reform come in many flavours. Some tout the benefits of more competition among schools; others aim to train more teachers and reduce class sizes. Still others plump for elaborate after-school programmes or for linking teachers’ pay to how well pupils do.

A relatively recent addition to this menu is the idea of paying students directly for performance. Boosters argue that pupils may fail to invest enough time and effort into education because the gains—better jobs and higher incomes—are nebulous and distant. Cash payments, on the other hand, reward good performance immediately. Link payments to test results or graduation rates, the argument goes, and test scores should increase and drop-out rates decline. Two new papers* describe the effect of such schemes in Israel and America. Their results will disappoint those who hope for a silver bullet. But they also suggest that cash payments may have their uses in some situations.

Joshua Angrist of the Massachusetts Institute of Technology and Victor Lavy of the Hebrew University in Jerusalem studied high-school students in 40 Israeli schools where few pupils went on to get their school-leaving certificate (the Bagrut). In half the schools students were offered a chance to earn nearly $1,450 if they passed all the tests and got the certificate. The economists found that completion rates in “payment schools” increased by about a third—but only for girls and mainly for those who needed to do only a tiny bit more to graduate.

In America Roland Fryer of Harvard University carried out an ambitious set of experiments involving 38,000 students who went to state-run schools in New York, Chicago, Dallas and Washington, DC. Over four-fifths were from poor families; nearly 90% were black or Hispanic. These are the schools that reformers in America most urgently need to fix. Students in inner-city schools do particularly poorly in national tests. Less than 20% of their 8th-graders (13-14-year-olds) have better-than-basic reading skills for their age. The national average is 29%.

In each city about half the participating schools were randomly selected to be ones where students received money; their progress was then compared with that of their peers in the other schools. Pupils in New York and Chicago were paid for test scores or grades. The children in Dallas and Washington, DC, were paid for specific tasks, like reading books or wearing uniforms.

The results of the experiments where scholastic performance was rewarded were uniformly disappointing. In New York fourth- and seventh-grade students in payment schools could earn up to $25 and $50 respectively, depending on their score on each of ten standard maths and reading tests. In Chicago ninth-grade students were rewarded on a sliding scale for good grades in five courses, including English, maths and science. Getting an “A” was worth $50; a “D” meant no money. In theory a student could earn up to $2,000 a year. Plenty of money was paid out, but Mr Fryer found absolutely no evidence that paying students led them to do better than their peers in the control schools. Neither girls nor boys gained, and it did not seem to matter how students had previously performed.

What explains this disappointing result? Some argue that the external push provided by money erodes an inherent love of learning. But participating students also took tests that measured how much they enjoyed studying. There was no indication that the payments affected those sentiments. Nor was it the case that students were uninterested in the programme.

Mr Fryer has a different explanation. Most would agree that school facilities, teachers’ skills, and the effort both students and teachers put in all matter. But how precisely these inputs are converted into a test score is a mystery, and without knowing which lever to pull, it is difficult to design an effective incentive scheme. But leaving it up to participants to find the best way to earn goodies will not work either if, as Mr Fryer believes, pupils have very little idea how to go about improving their own scores.
Grade expectations

When students in New York or Chicago were asked how they would earn the rewards on offer, they came up with all sorts of ideas about test-taking strategies, but not one mentioned reading the textbooks or doing practice questions. On the other hand, those whose performance improved in the Israeli experiment had clear ideas about how to go about making sure they graduated. They took more practice tests and were much more likely to attend free coaching sessions.

If students do not know how to improve their own performance, the best strategy may be to pick a simple task, reward pupils for doing it, and hope that this translates into higher grades. This was the approach Mr Fryer took in Dallas, where second-grade students were simply given $2 for every book they read if they passed a computerised comprehension test on it. Predictably this spurred them to read more books and improved their vocabularies. But it also improved their school grades substantially, although this is not what they were paid for. A year after the payments had stopped, students in the schools that had offered money were still outperforming those in control schools, although the gap had narrowed. It may have helped that the Dallas students were younger. Middle-school students in Washington, DC, gained little from being paid for inputs like attendance. But the results from Dallas suggest that payments can help at least some students get more out of school.

* “The Effects of High Stakes High School Achievement Awards”, by Joshua Angrist and Victor Lavy. Forthcoming in the American Economic Review. “Financial Incentives and Student Achievement”, by Roland G. Fryer junior. NBER Working Paper 15898, April 2010.