MISSISSIPPI MUDDLE: Score gains larger than reported!

WEDNESDAY, DECEMBER 18, 2019

Let's take a look at the record: In print editions, the New York Times wasn't especially delicate in its choice of words.

The Times was publishing an opinion column by Emily Hanford, senior education correspondent at American Public Media. And in her opinion column, Hanford was reporting some real information:

Over the course of the past six years (from 2013 through 2019), Mississippi's fourth graders have shown large score gains in reading! These gains have been recorded on the National Assessment of Educational Progress (Naep), the widely-praised "gold standard" of domestic educational testing.

"Mississippi is still the poorest state, but fourth graders there now read at the national average," Hanford wrote, referencing the state's average score on last year's Naep reading test.

"Mississippi’s fourth-grade reading scores are up by 10 points since 2013," Hanford went on to say, failing to offer a way to know if a gain of ten points should be seen as a lot or a little.

In fact, the score gain in question is slightly over 10.8 points—and that rounds off to eleven! And yes, if we credit those Naep data, that should be seen as a substantial, impressive gain.

According to a common but very rough rule of thumb, ten points on the Naep scale is often equated to one academic year. If we credit those Naep data, fourth-graders in Mississippi were roughly one full year ahead of their counterparts from 2013 on last year's reading test!

If real, that's a major gain. In print editions, the New York Times was none too delicate in the way it chose to headline the piece which was reporting this score gain.
.
Online, Hanford's column appears today under a highly flattering headline: "There Is a Right Way to Teach Reading, and Mississippi Knows It."

That headline flatters Mississippi, while possibly advancing a slightly peculiar thesis. In its print editions, though, the Times was a bit less delicate. In print editions on December 6, Hanford's column appeared beneath this somewhat indelicate head:

Perpetual Laggards Leap Ahead in Reading

Those perpetual laggards in Mississippi had staged a great leap forward! So it has sometimes tended to go when our self-assured Yankee tribe directs its perceptive gaze Southward.

In fairness, it isn't that the headline that day was "wrong." It was just perhaps a bit strong on the tribal aroma.

In fact, Mississippi, our poorest state, has long been one of the lowest-performing states on tests of reading and math. If the state's fourth-graders now match the national average in reading—if the state has made that much progress that fast—then that actually has been a great leap forward, one made by our laggard class!

Tomorrow, we'll start to look at Hanford's claims about the way this advance has been achieved. We find her thesis somewhat puzzling and somewhat depressing, though it may be perfectly accurate.

For today, we thought we'd fill in the background about Mississippi's ongoing gains on the Naep. In fact, the state's score gains are actually larger, and more widespread, than Hanford reports in her column.

How large have Mississippi's score gains been on the Naep? Today, we'll expand our field of view. We'll look at Mississippi's gains in both reading and math.

As we start, let's continue with fourth grade reading. How large have the score gains actually been?

As noted, Hanford reported a ten-point gain from 2013 to 2019, a gain which is really eleven. Most likely, she chose 2013 as her starting point because that was the state's worst performance on all recent Naep reading tests.

Having said that, whatever! How large have the gains really been?

As noted, the state's fourth-graders gained almost eleven points in reading over the six-year period to which Hanford referred. Nationwide, the fourth-grade reading score actually dropped by one point over that same period.

This allowed Mississippi to advance from twelve points below the national average to a state of virtual parity, all in just six years.

If real, that's a large advance! But Hanford is working with "aggregate" scores—with the average scores for all the fourth-graders of the state and the nation. If we "disaggregate" those scores—if we look at the gains recorded by the state's different demographic groups—the score gains, and the overall performance, are more impressive still.

Consider Mississippi's black fourth graders. Back in 2009, they scored slightly more than six points behind the nation's black kids, on average, on the Naep reading test.

By last year, that had changed. Last year, Mississippi's black fourth-graders outscored their black counterparts nationwide by something approaching six points:

Average scores, Grade 4 reading
Black kids, 2019 Naep
Mississippi: 208.61
United States: 202.96

Especially in such a low-income state, that's a striking advance and a striking performance. Mississippi's white fourth-graders matched their white peers across the nation last year, a surprising performance in a low-income state. But the state's black kids actually outperformed their peers nationwide, by a substantial margin.

Have we mentioned the fact that Mississippi is a low-income state? When we disaggregate by both income and race, the state's performance in fourth-grade reading again becomes more impressive.

How well did Mississippi's low-income fourth-graders perform on the Naep reading test? You're asking a very good question!

(Note! In federal education statistics, "low income" isn't a synonym for "poverty." Low-income kids are those who qualify for the federal lunch program. In theory, this means that their family incomes may be almost double the federal poverty rate. This is a very rough measure of income, but it's pretty much the only one we have.)

How well did low-income Mississippi kids perform on last year's Grade 4 reading test? Good lord! Low-income black kids outperformed their low-income peers nationwide by slightly more than nine points—by almost a full academic year:

Average scores, Grade 4 reading
Low-income black kids, 2019 Naep
Mississippi: 208.10
United States: 199.03

It wasn't just the black kids. Low-income white kids outscored their own low-income peers nationwide by slightly less than nine points. If we credit these Naep data, low-income kids in Mississippi are now strongly outscoring their low-income peers nationwide. And yes, this is a major change from the state's performance in the recent past.

Mississippi's fourth-graders did surprisingly well on last year's reading tests. After a bit of disaggregation, we'd say their performance was substantially stronger, and more surprising, than Hanford's column describes.

Having said that, let us also say this—Hanford only discussed the score gains on the Naep reading test. But when we look at the Naep math test, the impressive scores keep rolling along.

For today, how well did Mississippi's fourth-graders do on last year's Naep math test? In the aggregate, they outscored the nation by almost one point.

Given the poverty in the state, that's an impressive performance, and a break from the recent past. But when we disaggregate the state's fourth grade scores in math, the performance and score gains are just as strong as they were in reading, possibly somewhat better.

It isn't just that the state has "caught up" to the national average. In math as well as reading, different groups of Mississippi kids are whupping their peers nationwide:

Average scores, Grade 4 math
Black kids, 2019 Naep
Mississippi: 229.69
United States: 223.87

If we credit Mississippi's steady gains in math and reading on the Naep, something good seems to be happening in the state's public schools. You'd almost think a caring nation would want to know what's going on.

In fact, you'll never see such questions discussed in New York Times news reports. You'll never see such questions discussed on Tribal Cable, where the topic agenda goes like this:

Trump Trump Trump Trump Trump Trump Trump impeachment impeachment impeachment impeachment polls polls polls polls polls

In truth, no one cares about the kids of Mississippi, whether those kids are black or white, whether low-income or not. Hanford's column will come and go, having produced zero interest and even less debate and discussion.

Except here! Our discussion will shape up like this:

In her column, Hanford offered an explanation for the rise in Mississippi's reading scores. Her explanation may even be right—but if it is, she's describing a situation which is deeply depressing.

Are Mississippi's score gains real? If so, what has produced them?

Tomorrow, we'll start to look at Hanford's explanation—but you'll see this discussion nowhere else. In this, the age of Maddow and Trump, we can assure you of one thing:

No one in either tribe actually cares about any of this. Few facts could be more clear.

Tomorrow: The columnist's explanation

For all Naep data: For all Naep data, just start here.

From there, you're on your own.