INTERLUDE: What the heck is the NAEP?

FRIDAY, JUNE 23, 2023

Main Naep as opposed to Long-Term Trend Assessment: In yesterday morning's New York Times, Dana Goldstein penned a somewhat cogent report about the latest batch of test scores from the National Assessment of Educational Progress (Naep).

At various points, the report was well written. That said, the news was unmistakably bad.  Headline included, Goldstein's report began as shown:

What the New, Low Test Scores for 13-Year-Olds Say About U.S. Education Now

The math and reading performance of 13-year-olds in the United States has hit the lowest level in decades, according to test scores released today from the National Assessment of Educational Progress, the gold-standard federal exam.

The last time math performance was this low for 13-year-olds was in 1990. In reading, 2004.

Performance has fallen significantly since the 2019-2020 school year, when the coronavirus pandemic wrought havoc on the nation’s education system. But the downward trends reported today began years before the health crisis, raising questions about a decade of disappointing results for American students.

Without any question, the Covid pandemic seems to have affected public school performance in a substantial way. That said, Goldstein failed to distinguish between the two different testing programs which are administered as part of the Naep.

Over the course of the next week, we'll continue to look at the widely discussed Naep scores which have been recorded, in recent years, by kids in the Mississippi public schools. In hopes of eliminating possible confusion, let's describe the two (2) parallel testing programs administered within the Naep.

The so-called "Main Naep"

The so-called Main Naep has been in existence since 1990. As its name suggests, it's now considered to be the Naep's primary testing program.

Among other subjects, the Main Naep tests students in Grades 4, 8 and 12 in reading and math. Due to the size of the samples of students who get tested, the Main Naep is able to report reliable data for the nation as a whole, and for each of the fifty states.

In writing about Mississippi's public schools, the Associated Press was referring to that state's performance in recent years on the so-called Main Naep. Until a one-year delay in 2021 due to Covid, the Main Naep was being administered on a biannual basis.

The Long-Term Trend Assessment

Goldstein's report in the New York Times referred to brand new data from the Naep's parallel testing program, The Long-Term Trend Assessment. Like the Main Naep, this program tests students in reading and math—but it tests students who are 9, 13, and 17 years old, regardless of what grade they're in.

Key point—the so-called Long-Term Trend Assessment is older than the Main Naep! The program dates to the early 1970s, when the federal government began tracking student achievement. 

This original testing program is now administered more rarely than the Main Naep. As noted, Goldstein was reporting new results from the 2022-2023 school year.  Before that, the Long-Term Trend Assessment had last been administered in 2020, and in 2012 before that. 

For Kevin Drum's overview, click here.

When we return to Mississippi's kids, we'll be discussing the way they've performed in recent years on the Main Naep. That said, this passage from Goldstein's report applies to Naep testing in general:

GOLDSTEIN: In the highly decentralized American education system, NAEP is one of the few consistent tests given across states lines over many years, making the results easily comparable.

Scores on the exam do not result in any rewards or punishments for students, teachers or schools, making them especially useful for research purposes, since there are fewer incentives to cheat or teach to the test.

As compared to our various statewide testing programs, "there are fewer incentives to cheat or teach to the test" on the Naep?

We applaud Goldstein for her explicit reference to the outright "cheating" which has often occurred on less secure statewide tests. If anything, though, she may be understating the matter:

As far as we know, it isn't just that there are fewer incentives to cheat on the Naep. As far as we know, there are fewer opportunities for misguided teachers and principals to do so, given the way the Naep is administered.

We assume the worst about data from statewide tests. With a few caveats thrown in, we assume that the Naep's results are basically reliable.

That said, why have you never seen these matters fleshed out in more detail—in the New York Times, for example? 

Answer! Because no one actually cares about this, and no one ever will!

As always: According to Goldstein, "The last time math performance was this low for 13-year-olds was in 1990."

That's true if you don't disaggregate. If you make a few basic statistical adjustments, and if you also consult the Main Naep, less gloomy pictures emerge.

There has still been substantial loss, especially in the Covid years. But the picture isn't as thrillingly bad.