The accuracy of federal education data

by Jill Barshay

Correcting mistakes may be an essential part of a good education, but that doesn’t apply inside the branch of the U.S. government that compiles and keeps education statistics. Indeed, the National Center for Education Statistics (NCES) knowingly leaves in errors that are discovered two to three years later. And then this error-ridden data is used by education policy makers to make decisions. I recently learned about these revision deadlines from the person in charge of the education data, NCES Commissioner Jack Buckley. Buckley explained that the Integrated Postsecondary Education Data System data set, a.k.a. IPEDS, allows for one year of revisions (after the initial collection year) and then “locks” that year’s data forever. That’s been a frustration for for-profit universities who’ve been clamoring to retroactively revise their graduation rates upwards so that their students can remain eligible for federal student loans. Another major data collection, the Common Core of Data, is kept active for only three years, effectively cutting off revisions afterwards. The IPEDS data is self-reported and it may be wise to limit the ability of schools to game the data to their benefit. But the “locking down” of the data also means that more innocent mistakes are made that can never be fixed -- and then used for analysis by the public. At a May 2, 2013 session of the Education Writers Association California Watch, investigative reporter Erica Perez recounted how she repeatedly found mistakes in the federal databases when she phoned schools to verify the numbers. Contrast the Department of Education’s data practices with the Bureau of Labor Statistics inside the Department of Commerce, which never locks its data and allows for infinite revisions of its jobs and other economic data. (I verified this with the BLS). Why shouldn’t the education folks, like the economists, always correct an error when one is found? Commissioner Buckley, in an email, said that he sees a “tradeoff between getting it right the first time (since these data are used by policy makers and the public for annual decision making)” and “ allowing some flexibility for inevitable errors. Allowing schools to submit revisions for any year, any time is confusing for users and difficult for data collectors trying to maintain a complex process and keeping it timely and accurate.” It’s hard to make a blanket assessment of the quality of the federal education data. The Department of Commerce rushes data out to the public as quickly as possible, making it more error prone. The Department of Education takes a much longer time to verify and clean up the data before its first release. “We spend a lot of time and money on data quality. Beware of the fastest person to put data out,” said Buckley at an April 30, 2013 session of the American Educational Research Association’s annual meeting.