MISSISSIPPI'S MIRACLE: Do we really have a "Mississippi mirage?"

TUESDAY, JULY 4, 2023

Final thoughts on the latest miasma: By all accounts, the state of Mississippi has been working, very hard, to improve its public schools.

We applaud every person who has been involved in those ongoing efforts. That said, has the state engineered a "Mississippi miracle," as some observers have said?

Has "an education revolution" taken place? Has a "huge success story" occurred?

As we showed you yesterday, the answer is quite plainly no! The refutation of that inviting, feel-good claim looks exactly like this:

Average scores, Grade 4 reading
2022 Naep
Asian-American kids, U.S. public schools: 238.49
White kids, U.S. public schools: 226.03
White kids, Mississippi: 229.53
Black kids, Mississippi: 204.41
Lower-income black kids, Mississippi: 202.76 

Those scores display a yawning "race gap"—the kind of large achievement gap which was once understood to constitute the nation's public school problem.

Today, that original problem is disappeared as we blather about "huge success."

That said, our nation's journalistic and academic elites have long shown remarkably little regard for the lives and the interests of black kids. It's amazing to see the ease with which such kids can get kicked to the curb.

No, Virginia! Mississippi hasn't yet found the way to teach "the science of reading" to all its public school kids. That said, our journalists and experts have spent fifty years trying to wish this problem away. 

Claims of this miraculous "huge success story" are just the latest chapter in this fifty-year dodge. This "Schools That Work" story is deeply appealing. 

It's a bit of a zombie Storyline. It never quite goes away.

In fairness, Mississippi's Naep scores actually can look quite impressive when compared to those of most other states, at least on the fourth grade level. We close today with a question we've discussed earlier:

Could Mississippi's improved test scores result, in part or even in whole, from a "statistical mirage?"

To some extent, are we currently discussing a "Mississippi mirage?" As you may recall, that's the term Kevin Drum used when he first examined this topic roughly two weeks ago:

DRUM (6/24/23):  Mississippi's reforms included something called the "third-grade gate," which means holding back kids who can't pass a reading test at the end of the year. This is obviously going to improve scores for 4th graders, but it's a bit of a statistical mirage.

Drum thought the third grade retention policy would "obviously" improve Mississippi's fourth grade Naep scores. He also thought that improvement would be "a bit of a statistical mirage." 

Some commenters have had a hard time seeing the way this would work. In one last attempt at clarity, let's consider the dueling retention policies of two wholly imaginary American states.

We'll call our states State A and State B. Here's the phantasmagoric way how their retention policies would differ:

State A's retention policy

State A is a very laid back, rather traditional state. It promotes all its first graders to second grade—and this continues right on up the line.

When kids in State A have completed third grade, they all get promoted to fourth! There is zero retention in State A. Its third-graders all move ahead.

State B's retention policy

State B has adopted an extremely different approach. When state officials heard that Mississippi has achieved success by holding 9 percent of its third graders back, they decided to go all the way:

As a result, State B has decided to hold all its third graders back for an additional year before they proceed to fourth grade! They all spend an extra year in Grade 3 before moving on to Grade 4!

Stating the obvious, no states has ever adopted a policy like that which obtains in State B. Most likely, no state ever will take any such step.

That said, just imagine the state of play when it's time for kids from State A and State B to take the Grade 4 Naep tests. The state of play will be this:

In State A, all the Grade 4 kids will be "true age" fourth graders. They'll all be normal fourth grade age, and they'll all have had received four years of graded instruction (grades 1, 2, 3 and 4).

In State B, things will be different. After spending two years in third grade, all the fourth graders in State B will be normal fifth grade age. They all will have had five years of grade school instruction when they take the Grade 4 tests.

State B's fourth graders will all be one year older than their counterparts in State A. They will all have had one full additional year of public school instruction.

Would anyone expect State A's fourth graders to score as well as State B's kids? Presumably, the answer is no. Presumably, everyone would understand something else:

Presumably, people would understand that it would be very hard to compare those two states' Grade 4 test scores. 

Presumably, everyone would understand that State B's higher scores would involve a type of "statistical mirage." Presumably, everyone would see that State B had a built-in statistical advantage in the Grade 4 tests, due to its policy of holding all third graders back.

Presumably, this same principal obtains, to a lesser degree, in the case of Mississippi's Grade 4 Naep scores. Meanwhile, understand this significant fact:

As a matter of general policy, Mississippi has always held a lot of grade school kids back! 

It isn't just those lower-performing kids under the third grade retention policy. Headline included, here's a report from 2019, when Mississippi burst on the scene with its improved Grade 4 scores:

Mississippi rising? A partial explanation for its NAEP improvement is that it holds students back

[...]

In response to the legislature’s 2013 Literacy Based Promotion Act (LBPA), Mississippi schools retain a higher percentage of K–3 students than any other state...

The LBPA created a “third grade gate,” making success on the reading exit exam a requirement for fourth grade promotion. This isn’t a new idea of course. Florida is widely credited with starting the trend in 2003, and now sixteen states plus the District of Columbia have a reading proficiency requirement to pass into fourth grade.

But Mississippi has taken the concept further than others, with a retention rate higher than any other state. In 2018–19, according to state department of education reports, 8 percent of all Mississippi K–3 students were held back (up from 6.6 percent the prior year). This implies that over the four grades [K-3], as many as 32 percent of all Mississippi students are held back; a more reasonable estimate is closer to 20 to 25 percent, allowing for some to be held back twice.

Say what? "Over the [first] four" years in public school (K-3), "as many as 32 percent of all Mississippi students are held back?" 

So reported Todd Collins, writing for the Fordham Institute.

From kindergarten through Grade 3, as many as 32 percent of all Mississippi students [were being] held back! When compared to other states, that practice—right or wrong, wise or unwise—would gift Mississippi with an unusually old roster of fourth-grade students.

Fleshing out his basic point, Collins then offered a chart which showed how many kids Mississippi had held back in the 2018-2019 school year:

Percentage of kids held back
Mississippi, 2018-2019 school year
Kindergarten: 8.7%
Grade 1: 7.9%
Grade 2: 5.0%
Grade 3: 9.6%
Average: 7.8%

You're reading that correctly! In that school year, almost ten percent of third graders got held back in Mississippi. But so did almost nine percent of the state's kindergarten kids!

Rightly or wrongly, Mississippi was holding back a lot of its grade school kids. Collins continued as shown:

These retention levels are much higher than other states. The closest are Oklahoma at 6 percent and Alabama at 5 percent. Florida, probably the most well-known example, today holds back 4 percent of its K–3 students, including 8 percent of third graders. When it first enacted its retention policy in 2003–04, Florida’s third grade retention rose as high as 14 percent before steadily declining; it has risen again in recent years. The average for all states is about 3 percent; many states have retention rates of 2 percent or less.

As of the 2018-2019 school year, Mississippi was holding way more K-3 students back, as compared to most other states. Whether you approve of this practice or not, it presumably gives Mississippi a statistical advantage when its test scores are compared to those from other states, or from the nation as a whole.

Presumably, this statistical advantage would still obtain in Grade 8 testing, where Mississippi's Grade 8 population would be substantially older than those from other states.

None of this can tell us whether Mississippi's retention policies make sense. You may think that heavy Grade 3 retention makes superlative sense, or you may think it's a lousy idea. That isn't the issue here.

Whatever you think of grade school retention, Mississippi's heavy retention practices makes it harder to compare Mississippi's Naep scores to those of other states. On the simple basis of age and number of years in school, we're no longer comparing apples to apples.

Nicholas Kristof blew past this rather obvious statistical problem. So did his excited education expert.

We salute the efforts being made in Mississippi's public schools. But no miracle has taken place. Once again, we prove that by showing you this:

Average scores, Grade 8 reading
2022 Naep
Asian-American kids, U.S. public schools: 281.07
White kids, U.S. public schools: 267.11
Black kids, Mississippi: 240.37

Does that look like a "huge success story?" It looks like the problem to us!

Tomorrow: A few last thoughts on this (extremely important) topic