MISSISSIPPI'S KIDS: We don't agree with Kevin Drum...

MONDAY, JUNE 26, 2023

...concerning one basic point: We disagree with (the long-admired) Kevin Drum with respect to one basic point.

Before we revisit that basic point, let's recall why Nicholas Kristof claims that a revolution is underway in Mississippi's public schools.

Simply put, Mississippi's Grade 4 reading scores can look extremely good! For a full rundown, click here for last Thursday's report. 

Those reading scores can look very good. Here are three examples of what we're talking about:

Average scores, Grade 4 reading
Lower-income kids, 2022 Naep
U.S. public schools: 202.67
Mississippi: 211.74

Average scores, Grade 4 reading
Lower-income black kids, 2022 Naep
U.S. public schools: 193.42
Mississippi: 202.76

Average scores, Grade 4 reading
Lower-income white kids, 2022 Naep
U.S. public schools: 211.49
Mississippi: 224.45

For all Naep data, start here.

Holy cow! On the 2022 Naep reading test, Mississippi's lower income Grade 4 students outscored their counterparts across the U.S. by almost one academic year!  

Lower-income black kids also recorded a gap of that size when compared to their peers nationwide. Mississippi's lower income white kids were on the good end of a gap which was even larger.

On their face, those numbers can look extremely good. Something else is true, of course—an experienced, savvy observer won't automatically put his faith in such numbers.

It isn't that people cheat on the Naep. As far as we know, there's virtually no way to do so. Also, there are few incentives for anyone to do so, except perhaps on the level of a state superintendent.

That said, there are many ways for testing programs to produce misleading data. Any experienced person will automatically understand that—but through no particular fault of his own, Nicholas Kristof isn't an experienced education analyst.

Based upon his June 1 essay in the New York Times, Kristof pretty much accepted those Grade 4 reading scores at face value. In his essay, he said those apparently impressive scores were part of "a revolution" in Mississippi, a revolution based on new, improved ways of employing "the science of reading."

Having said that, how (perhaps slightly) odd! As you can see, Mississippi's Grade 4 scores on the Naep math test were almost equally impressive;

Average scores, Grade 4 math
Lower-income kids, 2022 Naep
National public schools: 222.73
Mississippi: 229.13

Average scores, Grade 4 math
Lower-income black kids, 2022 Naep
National public schools: 212.16
Mississippi: 218.80

Average scores, Grade 4 math
Lower-income white kids, 2022 Naep
National public schools: 232.50
Mississippi: 241.74

Holy cow all over again! As it turns out, Mississippi's lower-income Grade 4 kids outscored their counterparts in math by margins which were also quite impressive.

No one claims that Mississippi has found some new, improved way to teach grade school math. But the state's large array of lower-income kids outscored their counterparts nationwide in Grade 4 math as well, by a margin roughly two-thirds the size of their margin in reading.

Friend, are you inclined to think that data like those should be taken at face value? Long ago, we ourselves might have reacted that way—but long experience has taught us that (pleasing) data like these can be misleading, even where all parties are behaving in total good faith.

Example: 

Way back in 2006, the Washington Post ran a report across the top of its front page about the outstanding test scores at little, low-income Maury Elementary in Alexandria, Virginia. The banner headline said this: A Study in Pride, Progress.

After examining the statistical fine print, we showed that Maury's third graders actually had the second lowest reading scores of any school in the whole state of Virginia! In that case, the gross misdirection tracked back to several ludicrous administrative and statistical procedures which had been engineered within the state's education department. 

For our interview with the president of the Virginia Department of Education, you can just click here. Everyone agreed that the Post had been grossly misled by Mury's reported scores, but so were Post readers, of course. In our own direct and varied experience, this sort of thing had already been going on for decades at that point.

There are many ways for test score data to be significantly misleading. However pleasing such test scores may seem to be, data like those from Mississippi actually can be misleading. 

So can drive-by claims of "miracles" or "revolutions" from inexperienced journalists—from well-intentioned journalists who spend a day or two visiting some high-scoring school or school system, all the while believing the various things they get told.

If we actually care about the lives of this nation's low-income kids, we'll be very careful about the conclusions we draw from such pleasing data. Truth to tell, our tribunes in the press corps rarely exhibit such care, and our "education experts" simply never speak up when the state of play in the pubic schools is being misdescribed.

Question:

If no one cheated in any way to produce Mississippi's Grade 4 scores, how could those impressive scores possibly be misleading? 

You're asking a very good question! Luckily, we explained the possible problem with Mississippi's Naep scores way back on June 5, in this somewhat detailed report. 

We explained the possible statistical problem in detail that day. Here's the way the long-admired Drum has now (quite cogently) thumbnailed that possible statistical problem:

DRUM (6/23/23): Bob is skeptical of these scores for another reason: Mississippi's reforms included something called the "third-grade gate," which means holding back kids who can't pass a reading test at the end of the year. This is obviously going to improve scores for 4th graders, but it's a bit of a statistical mirage.

On the other hand, only 9% of Mississippi's third-graders are held back, so the effect is probably modest. 

It's just as Kevin says! As we explained on June 5, holding back an unusual number of lower-performing third-graders will almost surely "improve [the state of Mississippi's average] scores for 4th graders."

Also this:

To the extent that Mississippi's scores improve for that reason, the improvement in Grade 4 scores is, in fact, "a statistical mirage," just as Kevin says. 

Most simply put, a state which makes a lot of kids repeat third grade will almost surely have a statistical advantage over other states which don't. It's hard to make a valid comparison between states which hold a lot of kids back as opposed to states which don't.

In that passage, Kevin is describing the statistical problem we detailed on June 5. Having said that, we don't necessarily agree with his claim that the statistical advantage gained by Mississippi will "probably [be] modest."

As we showed you on June 5, the lowest-performing ten percent of fourth graders produce very low scores on Grade 4 Naep tests. Conceivably, eliminating those lowest performers from the population which gets tested could improve a state's average score by a fair amount, especially in the first year of the third grade retention program.

Does Mississippi's Grade 3 retention program explain those high Grade 4 Naep scores? We'll answer your question in this time-honored way:

Goofus doesn't want to bothered with such boring, tedious questions. Gallant insists that we should check the possibility out.

Putting it a different way, we would tell you this:

People who care about low-income kids won't be inclined to accept such feel-good test score data at face value. That said, alas:

For the past fifty years, our education journalists, such as they are, have tended to peddle the "feel good" stories, and they have then walked away. As for our alleged "education experts," they have tended to slumber and snore every darn step of the way:

Those alleged education experts! They slept through the endless cheating scandals which we first described, in the Baltimore Sun, back in the 1970s. Decades later, it fell to newspapers like USA Today to report the astonishing cheating scandals our alleged education experts had agreed to ignore.

Also this:

As we endlessly noted at this site, Naep scores improved for all demographic groups, on a nationwide basis, all through the 1990s and the 2000s—but so what? Even as those "disaggregated" Naep scores continued to climb for all major groups, our education experts refused to inform the public of this important fact, even as our hapless mainstream journalists kept insisting that "nothing has worked" in our public schools.

It's hard to have sufficient contempt for the decades of silence from these expert lambs. That said, this is the way our national discourse actually works, even among the academics and journalists who comprise our blue tribe's honored elite.

Our discourse works this way with respect to the public schools, but also with respect to a wide array of other subject areas. When it comes to the public schools, our education experts refuse to speak up. So too with massively-paid "cable news" stars like Rachel, Chris and Lawrence. You'll never hear about Mississippi's kids on our tribe's cable news shows.

In short, our blue tribe's elites are, at best, just extremely careless. At their worst, they don't seem to care about Mississippi's low-income kids, and the odds say they never will.

We disagree with Kevin Drum about the possible size of the "statistical mirage" created by that third-grade retention program. Before we liberals pleasure ourselves by saying a "revolution" is underway, we need to spend a bit more time pondering those somewhat unlikely statistics.

Is a revolution really underway in Mississippi's schools? Has a "Mississippi miracle" taken place in recent years?

We've seen this kind of careless journalistic behavior many times over the past fifty years. Our journalists feed us a pleasing tale about some vast success somewhere. As a general matter, we don't discuss our low-income schools for the next quite a few years.

In our next report, we'll show you what Kristof's revolution looks like on the Grade 8 level. Spoiler alert:

On the Grade 8 level in Mississippi, Kristof's "revolution" looks like a revolution which has failed. Or did that alleged revolution simply never happen? Our blue tribe's history is quite clear here. It has been a very long time since we actually seemed to care.

Mississippi is full of good decent (low income) kids with good decent (low income) parents. As Langston Hughes once put it, life for them ain't been no crystal stair.

On the highest levels of our own tribe's discourse, no one actually cares about such children and such parents. Over the course of the past fifty years, the evidence in support of that claim has been surprising but clear:

Within the elites of our self-impressed tribe, no one cares about low-income kids. Also, the odds are quite good that nobody ever will.

Why don't our tribe's major stars discuss the lives of Mississippi's kids? For those who are willing to see, the answer is blindingly clear.

Tomorrow or Wednesday: Mississippi's Grade 8 scores

After that: The expert's tale