Individual U.S. Schools Take Part in PISA Pilot

By Erik W. Robelen

Premium article access courtesy of Edweek.org.

More than 100 U.S. schools will soon find out how they stack up against the world, and perhaps take away some valuable insights to improve their practices.

The schools, spanning 20 states, are participating in a new pilot project in which 15-year-olds take a 2½-hour test based on a high-profile global assessment best known by the acronym PISA. The results will be comparable to those of dozens of nations that take part in the Program for International Student Assessment­, including some of America’s top economic competitors.

The paper-and-pencil exam seeks to measure students’ higher-order thinking skills in reading, mathematics, and science, with a focus—as with the main PISA—on applying knowledge to real-world situations.

Led by the Paris-based Organization for Economic Cooperation and Development, which oversees PISA, the new initiative is being implemented with support from several private U.S. grantmakers and partner organizations.

“We just keep hearing how bad U.S. schools are and that we don’t measure up,” said James R. Hogeboom, the superintendent of the 10,700-student Lucia Mar district in California, in explaining his system’s system’s plans to have one of its high schools take part. “Well, is that true or not true? And what do we need to improve our kids’ critical-thinking skills? I’d love to know that.”

The pilot test is scheduled to take place this month and next, with results expected back to local schools and districts by September. It’s up to local officials to decide whether the results are made public.

The initiative is separate from the United States’ planned participation in the upcoming administration of the regular PISA, scheduled for next fall, which provides national-level data.

The pilot comes as the Obama administration, in its fiscal 2013 budget request, proposed spending $6 million to create a separate PISA pilot initiative that would allow individual states to compare their achievement against other nations’. ("NAEP Would Slip, PISA Gain in 2013 Budget Plan," Feb. 29, 2012.)

Broad Interest

Many education, business, and political leaders have lamented the United States’ standing in international achievement comparisons and have argued that the situation spells trouble for the nation’s economic competitiveness.

In the latest round of PISA, in 2009, U.S. achievement in science and reading was about average for the 34 member nations in the OECD. In math, U.S. students’ achievement was below average.

The United States ranked in the same statistical category for math as 12 other nations, including the Czech Republic, Hungary, Ireland, and Spain. Meanwhile, 17 OECD nations had measurably higher scores, including Canada, Germany, Japan, and South Korea.

In all, 60 nations participated in PISA in 2009, as well as several large, non-national education systems, including Shanghai, China, which is a municipal province.

The new pilot brings together 104 American schools, nearly all of them public, to participate in the PISA-Based Test for Schools, from Arizona, California, New York, Ohio, and Rhode Island, among other states. In addition, 18 schools from the United Kingdom will take part.

Organizers declined to provide a complete list of schools, saying it’s up to the schools and their districts to decide if they want to make their involvement public.

Private funders supporting the work include Bloomberg Philanthropies, the Carnegie Corporation of New York, the Kern Family Foundation, the National Public Education Support Fund, and the William and Flora Hewlett Foundation. (The Hewlett and Carnegie foundations provide financial support for some of Education Week’s news coverage.)

The Rhode Island education department issued a press release in March on plans for six of its public schools to participate.

“We’re very keen on understanding how our performance compares to international benchmarks, because our goal is to prepare our students for success in the 21st-century global economy. As clichéd as it sounds, it’s really, really true,” Deborah Gist, the state’s education commissioner, said in an interview.

The Rhode Island schools are a mix and not just high-fliers.

“[They] ... range from schools identified as persistently low-achieving all the way up to those with really strong assessment results,” Ms. Gist said.

Patti Dicenso, the secondary performance officer for the 8,600-student Pawtucket district in Rhode Island, which has three high schools signed up, said her system jumped at the chance.The U.S. schools taking part are not considered a statistically representative sample, but organizers say they are intended to reflect a diverse mix of student demographics, achievement, and locations.

“It’s a wonderful opportunity for us to really feel like we’re comparing ourselves in a more global way,” she said, “and reaching out beyond our own boundaries here.”

Seeking Improvement

Organizers and participants emphasized, however, that the pilot is not simply a way to see how they compare, but also to assist in setting a course for improvement.

“The key thing is to learn what we need to do to help our kids become more competitive in critical-thinking skills,” said Mr. Hogeboom from the Lucia Mar district. “Really, this is a catalyst for reflection and discussion on what we’re doing at the school level.”

Several organizers suggested that some participating schools may come together later to compare notes about the experience and what to take away from their results.

The initiative is separate from the main PISA, which involves assessments every three years. While PISA is intended to provide aggregate national results for international comparison and to inform policy discussions, the PISA-Based Test for Schools is “designed to provide school-level results for benchmarking and school improvement purposes,” according to an OECD overview.

Once the pilot phase is completed, the OECD will consider with its member countries taking steps to make the new PISA-based assessment available, starting in 2013, to virtually any school or school district that wishes to take the exam. It could eventually be offered in multiple languages so that schools in a variety of countries could participate.

That said, this is not the first time school-level reports will be provided on PISA. In fact, the National Center for Education Statistics, part of the U.S. Department of Education, offered schools that were part of the U.S. sample in 2009 the option, for the first time, of getting such information. Those confidential school-level reports enabled schools to compare their performance against the United States’ and other participating nations’, as well as to similar schools in this country, and to some extent around the globe, based on different school characteristics. However, schools had to meet requirements for sample size and participation to get the reports.

The NCES, which is currently recruiting schools for the next round of national PISA testing in the fall, will once again make available such school-level reports to participants.

Whose Crisis?

With the PISA-based pilot test, the school reports will provide a performance summary in reading, math, and science comparable to the PISA scales, with some breakdowns by subgroups, if enough students are tested at the school. They also will contain comparative tables that allow for benchmarking the school results in the context of PISA 2009 scores for participating countries and economies. In addition, the reports will consider the socioeconomic status of students so that the school might better understand how its achievement results compare with those of schools in other countries with similar socioeconomic profiles.

Jon Schnur, the executive director of a new nonprofit organization called America Achieves, which is the U.S.-based coordinator for the PISA pilot, said one value of providing school-level data is to move beyond national averages that may be easier for communities to dismiss.

“It often seems like someone else’s crisis,” he said, “and doesn’t feel like your own.”

May 4th,2012