Supplemental: Can NAEP scores be trusted?

TUESDAY, JUNE 3, 2014

Ruminations on the so-called gold standard: We’ll be looking at NAEP scores for the next several weeks during our series, “Our month of the gaps.”

This raises a basic question: Can NAEP scores be trusted?

We’d be inclined to say yes, with a few reservations.

In recent years, a great deal of fraud has occurred in the nation’s state-by-state, “high stakes” testing programs. For what it’s worth, no one has been on the cheating beat any longer than we have.

We first approached the Baltimore Sun about outright cheating on standardized tests in 1971 or 1972. We’ve pursued a series of other such episodes over the past forty years.

Cheating scandals have been common of late. With that in mind, can NAEP scores be trusted?

We’d be inclined to say yes.

In its 43 years of operation, the NAEP has never been a high-stakes test. Teachers and principals have nothing at stake in the results of these tests.

Beyond that, the NAEP is run by competent people, and it’s adequately funded. Often, this hasn’t seemed to be the case with our state-run tests.

For these reasons, the NAEP has always been described as the “gold standard” of domestic testing. We do have one potential concern.

In the last decade, the NAEP has emerged into the public eye. As a result, state superintendents do have something to gain, or something to lose, if a state does unusually well on the NAEP, or unusually poorly.

Can state superintendents put their thumbs on the scale as the NAEP selects its statewide samples of students for testing? If so, someone is going to monkey around with the statewide sample at some point—or someone already has.

In recent years, the state of Maryland has eliminated a very high number of students from the testing, part of a state’s prerogative under certain technical guidelines. The state scored well with its sample of students pared down in this way. This produced little discussion.

The nation’s press corps should have examined these questions long ago. In truth, little real reporting is done by the nation’s education reporters. They mainly exist to recite elite scripts about the stagnant scores in our failing schools—scripts which fly in the face of the actual data emerging from the NAEP.

Everybody praises the NAEP—and no one reports what the NAEP data show! We’ll be using NAEP data for the next several weeks.

They’re the only real data we have.