MISSISSIPPI MUDDLE: What explains Mississippi's score gains?

THURSDAY, DECEMBER 19, 2019

Scribe's answer should break all our hearts: In the immortal words of the New York Times, the "perpetual laggards" in Mississippi have recorded large score gains in recent years in Grade 4 reading and math.

The gains in question have been recorded on the National Assessment of Educational Progress (the Naep), the widely-praised "gold standard" of domestic educational testing.

On December 6, this information was reported—where else?—in a New York Times opinion column. In print editions, the column appeared beneath a snide headline which bore the immortal words we have quoted above. See yesterday's report.

Indelicate language to the side, how big have those score gains been? The column was written by Emily Hanford, an education reporter for American Public Media. She only discussed the gains in reading, although, as we noted yesterday, Mississippi's gains in math in recent years have been just as large:

HANFORD (12/6/19): New results on the National Assessment of Educational Progress, a standardized test given every two years to measure fourth- and eighth-grade achievement in reading and math, show that Mississippi made more progress [from 2017 to 2019] than any other state.

The state’s performance in reading was especially notable...Fourth graders in Mississippi are now on par with the national average, reading as well or better than pupils in California, Texas, Michigan and 18 other states.

[...]

For years, everyone assumed Mississippi was at the bottom in reading because it was the poorest state in the nation. Mississippi is still the poorest state, but fourth graders there now read at the national average. While every other state’s fourth graders made no significant progress in reading on this year’s test, or lost ground, Mississippi’s fourth-grade reading scores are up by 10 points since 2013...

In fact, Mississippi's Grade 4 reading score is up by 10.8 points over that six-year span, and that rounds off to eleven.

Judged by a conventional though very rough rule of thumb, this suggests that Mississippi's fourth graders are now performing one full academic year ahead of their fourth-grade counterparts from the year 2013.

If true, that's remarkable progress. And as we noted yesterday, Mississippi's performance is even more impressive if you disaggregate scores on the basis of income and race:

Black fourth-graders in Mississippi didn't just match the national average as compared to their peers nationwide. They outperformed their peers nationwide by almost six points on the 2019 Naep reading test.

Among Mississippi's low-income kids, the achievement is even more striking. The state's low-income black fourth-graders outscored their peers nationwide by more than nine points—roughly speaking, by almost a full academic year. Meanwhile, the state's low-income white kids outscored their counterparts nationwide by 8.6 points.

On their face, those scores are quite impressive. They lead us to an obvious question:

In a state populated by so many laggards, what explains the large score gains, and the impressive overall performance?

Why are kids in Mississippi outscoring their peers nationwide? In her recent New York Times column, Hanford sought to answer that question, though only with respect to the gain in the state's reading scores.

As noted, Mississippi's gains in Grade 4 math have been equally large. We'll restrict ourselves to reading today, in line with Hanford's focus.

Why have Mississippi fourth graders showed large score gains in reading? Why are the state's racial/economic groups substantially outperforming their counterparts nationwide?

Hanford offers a somewhat high-fallutin' explanation. It's previewed in the pair of headlines which adorn her column online:

There Is a Right Way to Teach Reading, and Mississippi Knows It
The state’s reliance on cognitive science explains why.

Mississippi's public schools have been relying on cognitive science? What type of "cognitive science" can Hanford possibly have in mind? And can that be the explanation for the score gains in question?

What explains Mississippi's performance? To the extent that Hanford's explanation may be true, it should break all our hearts.

Below, you see the start of her explanation. We'll simplify later on:

HANFORD: What’s up in Mississippi? There’s no way to know for sure what causes increases in test scores, but Mississippi has been doing something notable: making sure all of its teachers understand the science of reading.

Yes, there is a science to how people read. For the past several decades, in labs and classrooms all over the world, scientists have been studying how skilled reading works, what children need to learn to become skilled readers, and what’s going on when students struggle. Reading is probably the most studied aspect of human learning.

But a lot of teachers don’t know this science. In 2013, legislators in Mississippi provided funding to start training the state’s teachers in the science of reading.

In 2013, Mississippi began "training [its] teachers in the science of reading," Hanford says at the end of that passage. From there, she proceeds to a somewhat wordy explanation of what that "cognitive science" tells us—though we'll note that, just as she never used the word "laggards," she also never employed the somewhat pompous term, "cognitive science."

Leave that to the side! Hanford does refer to "the science of reading," and she does strongly suggest that Mississippi's recent reliance on this science explains the rise in its reading scores—a rise she actually understates in the several ways we've discussed.

For our money, Hanford's account of "the science of reading" is hugely underwhelming. You can read her lengthy account for yourself, but this column from October 2018 makes her basic meaning fairly clear:

Hanford seems to be saying that Mississippi began teaching phonics in its schools, "and that has made all the difference," at least to the extent that such things can be explained. We'll explore that earlier column tomorrow, but that's pretty much where it leads.

Mississippi began teaching phonics? On that basis, Mississippi's low-income kid are now strongly outperforming their low-income counterparts nationwide?

Once again, we'll note the fact that the state's math scores have risen just as much as its reading scores have. But to what extent could Hanford's explanation possibly be accurate? Mississippi began teaching phonics, and that explains its surprising reading scores?

If true, that explanation should break all our hearts. We complete our rumination tomorrow.

Tomorrow: "It’s not just ignorance. There’s active resistance to the science, too."

Fun with numbers: Kevin Drum has been doing a bit of debunking of late. We can't tell you why.

With regard to those Mississippi scores, he offers this analysis quoted below. He does so as part of a wider critique with which we would generally agree:

DRUM (12/18/19): Mississippi’s 4th grade reading scores have been going up steadily since 1992. They increased eight points between 2002 and 2009 and then ten points between 2013 and 2019.

Fun with cherry-picking? The 2009 score which Drum uses as a benchmark was a bit of an outlier. It was the highest score the state had ever recorded in fourth grade reading, and the highest score the state would record until 2015.

For that reason, while Drum's statement is perfectly accurate, this presentation would be perfectly accurate too:

Mississippi’s 4th grade reading scores have been going up steadily since 1992, but only slowly until recent years. They increased just 5.7 points between 2002 and 2013, but then increased by 10.8 points between 2013 and 2019.

Putting it yet another way, Mississippi gained just 9 points from 1992 to 2013, a period of 21 years. The state then gained almost 11 points over the next six years. For what it's worth, that 11-point gain was accomplished as the national average was dropping by one point.

We agree with Drum when he finds Hanford's overall presentation underwhelming. That said, Mississippi's surprising scores do seem to call for an explanation, especially after adjustment for income and race.

Concerning our response to Drum on the Grade 12 Naep, that is yet to come. For today, Chris Cuomo's weird meltdown concerning an alleged increase in crime has earned our afternoon space.

Gloom about schools never sleeps. So too with strange rants about crime.