INTERLUDE: What the heck is the NAEP?

FRIDAY, JUNE 23, 2023

Main Naep as opposed to Long-Term Trend Assessment: In yesterday morning's New York Times, Dana Goldstein penned a somewhat cogent report about the latest batch of test scores from the National Assessment of Educational Progress (Naep).

At various points, the report was well written. That said, the news was unmistakably bad.  Headline included, Goldstein's report began as shown:

What the New, Low Test Scores for 13-Year-Olds Say About U.S. Education Now

The math and reading performance of 13-year-olds in the United States has hit the lowest level in decades, according to test scores released today from the National Assessment of Educational Progress, the gold-standard federal exam.

The last time math performance was this low for 13-year-olds was in 1990. In reading, 2004.

Performance has fallen significantly since the 2019-2020 school year, when the coronavirus pandemic wrought havoc on the nation’s education system. But the downward trends reported today began years before the health crisis, raising questions about a decade of disappointing results for American students.

Without any question, the Covid pandemic seems to have affected public school performance in a substantial way. That said, Goldstein failed to distinguish between the two different testing programs which are administered as part of the Naep.

Over the course of the next week, we'll continue to look at the widely discussed Naep scores which have been recorded, in recent years, by kids in the Mississippi public schools. In hopes of eliminating possible confusion, let's describe the two (2) parallel testing programs administered within the Naep.

The so-called "Main Naep"

The so-called Main Naep has been in existence since 1990. As its name suggests, it's now considered to be the Naep's primary testing program.

Among other subjects, the Main Naep tests students in Grades 4, 8 and 12 in reading and math. Due to the size of the samples of students who get tested, the Main Naep is able to report reliable data for the nation as a whole, and for each of the fifty states.

In writing about Mississippi's public schools, the Associated Press was referring to that state's performance in recent years on the so-called Main Naep. Until a one-year delay in 2021 due to Covid, the Main Naep was being administered on a biannual basis.

The Long-Term Trend Assessment

Goldstein's report in the New York Times referred to brand new data from the Naep's parallel testing program, The Long-Term Trend Assessment. Like the Main Naep, this program tests students in reading and math—but it tests students who are 9, 13, and 17 years old, regardless of what grade they're in.

Key point—the so-called Long-Term Trend Assessment is older than the Main Naep! The program dates to the early 1970s, when the federal government began tracking student achievement. 

This original testing program is now administered more rarely than the Main Naep. As noted, Goldstein was reporting new results from the 2022-2023 school year.  Before that, the Long-Term Trend Assessment had last been administered in 2020, and in 2012 before that. 

For Kevin Drum's overview, click here.

When we return to Mississippi's kids, we'll be discussing the way they've performed in recent years on the Main Naep. That said, this passage from Goldstein's report applies to Naep testing in general:

GOLDSTEIN: In the highly decentralized American education system, NAEP is one of the few consistent tests given across states lines over many years, making the results easily comparable.

Scores on the exam do not result in any rewards or punishments for students, teachers or schools, making them especially useful for research purposes, since there are fewer incentives to cheat or teach to the test.

As compared to our various statewide testing programs, "there are fewer incentives to cheat or teach to the test" on the Naep?

We applaud Goldstein for her explicit reference to the outright "cheating" which has often occurred on less secure statewide tests. If anything, though, she may be understating the matter:

As far as we know, it isn't just that there are fewer incentives to cheat on the Naep. As far as we know, there are fewer opportunities for misguided teachers and principals to do so, given the way the Naep is administered.

We assume the worst about data from statewide tests. With a few caveats thrown in, we assume that the Naep's results are basically reliable.

That said, why have you never seen these matters fleshed out in more detail—in the New York Times, for example? 

Answer! Because no one actually cares about this, and no one ever will!

As always: According to Goldstein, "The last time math performance was this low for 13-year-olds was in 1990."

That's true if you don't disaggregate. If you make a few basic statistical adjustments, and if you also consult the Main Naep, less gloomy pictures emerge.

There has still been substantial loss, especially in the Covid years. But the picture isn't as thrillingly bad.


  1. It would be interesting to see a case made that 13 year olds in 1990 were intellectually inferior and became incompetent workers and citizens.

    This is the the kind of weird and useless analysis suggested by the nonsense seen daily here.

    1. Drum makes that argument about lead levels, linking to crime rates.

    2. That’s not remotely a similar argument.

  2. Bruce Ismay squeezed his stinking arse into a lifeboat; Stockton Rush died with his passengers.

    1. Rush didn’t choose to die that way. If a lifeboat had been available in his situation, he would be on it too.

  3. Conflation is the merging of two or more sets of information, texts, ideas or opinions into one, often in error.
    " ... why have you never seen these matters fleshed out in more detail—in the New York Times, for example?"
    "Because no one actually cares about this, and no one ever will!"

    1. Conflation is very different from inflation.

    2. Someone can care about education without needing to know the nerdy details of a specific test. We have experts to digest those and give us a summary.

    3. Essentially inflation is always profiteering.

  4. Somerby says NAEP results are reliable and yet he has been treating the MS results with suspicion for a month now.

    1. Somerby, the Inspector Clouseau of right wing blogging.

  5. No one is thrilled when results are bad. Is Somerby? If not, why does he say so?

  6. Liberals don't dare disaggregate. If they acknowledge that black are 3 - 5 years behind whites and Asians in school, on average, that would destroy the liberal narrative that racism is why black earnings lag whites and Asians.

    Of course, the liberal focus on racism, the wrong diagnosis, rather than education means that liberal policies don't help blacks catch up. But, that's OK with liberals. As Bob often points out, they don't really care about blacks.

    1. Your argument is invalid. They can say black kids are behind because of racism.

    2. All blacks are not 3-5 years behind. Don’t confuse means with individuals. There are more very low scores among blacks than whites which drag their average down. Why? Because there is more poverty, early childhood disadvantage, unaddressed health problems, less enrichment and early learning experiences in the home. That is not OK with liberals. Neither is treating all black kids as if they are 3-5 grades (not years) behind white kids. Racism prevents individual black kids from being hired and paid based on what each person can do, not stereotypes of being 3-5 years worse than a white job applicant.

      When black kids learn to read in MS, Somerby is first in line to disparage their improvement. He is the last person to say liberals don’t care when he himself attacks the very schools who have worked hard to help all the kids in MS, including black kids.

      This comment is beneath you, David.

    3. @9:03 - I am well aware of the difference between a mean and an individual. That's why I wrote "On average."

      Your list of causes is not based on specific research. I don't know whether your list is really the main reasons.

      Your list makes the problem sound almost hopeless. The items you mention will not be fixed for many, many years.

      I am more optimistic than you are. Other poor groups did catch up educationally. Sometimes the cure is not simply reversing the cause. I feel that there are ways to improve black education without solving the various problems you list. A good start IMO is better discipline.

    4. Yes, my list is based on sociology. The best way to close the gap is to fix poverty. You can’t beat that out of black kids. The pandemic payments helped a lot but coincided with school disruption.

    5. David’s assertion is valid…if you consider Blacks genetically inferior.

  7. Does this have anything to do with he Kristoff/ Mississippi story Bob has been beating to death?

    1. Two elitists that lean into neoliberalism and meritocracy, fighting among themselves.

      Can’t we all get along?

    2. I’m going to take that as a “no.”

  8. Daily Kos has modified its editorial policies to prevent false information in its stories. Among the changes is this:

    "We’ll be reading your comments closely, to see if there’s anything we might’ve missed that requires adjusting."

    Somerby should adopt such a policy himself. When he gets something wrong or hasn't provided enough backup for something he has posted, it is typically picked up in the comments. If Somerby were to respond to his comments, at least by think about and reevaluating his own information, he could prevent this blog from being a source of disinformation.

    A serious commitment to disseminating only reliable information is needed during a time when deliberate fake info is being spread for political purposes. Somerby needs to rethink his own policy of never reading his comments.

  9. Here is an example of media criticism. You may notice that it is very different than the "musing" that goes on here at Somerby's blog:

    After reproducing an example of an article from Fox, Steve M. says:

    "In that last example, AP's headline is "Louisiana House Passes Bill to Ban Gender-Affirming Care for Minors." The story's second paragraph says that bill prevents minors from receiving care "such as hormone treatments, gender reassignment surgery or puberty-blocking drugs." At Fox, the second paragraph also includes the list of prohibited procedures, but the headline -- which is all many readers will notice -- is "Louisiana House Passes Bill Prohibiting Doctors from Performing Sex Reassignment Surgeries on Minors," as if other interventions are unaffected.

    Sending wire service copy to an Orwellian team of rewriters is an old tradition at Fox. Two decades ago, when I started this blog, I wrote several posts about Fox's practice of replacing the phrase "suicide bomber" with "homicide bomber." Any idiot could tell you that what's makes suicide bombings especially horrifying is the suicide aspect -- we're wired to anticipate that a person standing in our midst won't willfully self-destruct -- but Fox didn't think the term "suicide bomber" made the perps seem evil enough. Also, Ari Fleischer and others in the Bush administration had adopted the term, and Fox considered itself the administration's other press office. At Fox, this led to ridiculous headlines such as "Homicide Bomber Kills One in Israel." (Well, duh -- if someone was deliberately killed, we know it was a homicide.) And as Atrios noted at the time, George W. Bush himself didn't even follow the practice -- he killed these killers "suiciders." But the language was changed at Fox, because freedom."

  10. The infectious grooves and catchy melodies worm their way into my head, making me wanna bust a move even when I'm not playing. fnf is my soundtrack to life.