TUESDAY, MAY 30, 2023
Is everything as it seems? Last week, we explored a topic we called THE UNDISCUSSED.
This week, we're discussing THE SEARCH.
It's important to remain clear about the trigger for that current search. In the present circumstance, here's the specific question we're trying to answer:
We're trying to learn if everything is at it seems with the so-called "Mississippi miracle"—the subject of this AP report back on May 17.
The AP report dealt with Mississippi's performance in Grade 4 reading in last year's administration of the National Assessment of Educational Progress (Naep). According to the AP report, the so-called Mississippi miracle consists in such things as these:
In its lengthy piece, the AP correctly reports that Mississippi "went from being ranked the second-worst state in 2013 for [Grade 4] reading to 21st in 2022." Indeed, Mississippi's fourth graders slightly outscored the national average on this reading test.
The AP also reports that Mississippi has achieved "promising gains for low-income kids in particular."
More precisely, the AP correctly notes that Mississippi was the second highest scoring state in the nation in that category on the Grade 4 reading test. Remarkably, Mississippi's lower-income kids outscored the national average on this Grade 4 test by almost one full academic year.
The AP reported such data correctly, but is everything as it seems with these heartening average scores? We ask the same question about Mississippi's status as the fifth highest-performing state on the Grade 4 test among the nation's black kids:
Average scores, black students
Naep, Grade 4 reading, 2022
Washington state: 209.79
New Jersey: 203.42
U.S. public schools: 198.12
We showed you those statistics yesterday. Mississippi's black fourth graders produced the fifth highest average score in the nation, among the 39 states with a large enough number of black kids to produce a statistically useful score. They outscored the national average among their peers by more than half a year.
Those are some of the data behind what the AP headlined as the "Mississippi miracle."
Summarizing, Mississippi outperformed the national average among fourth-black graders as a group. It ranked especially high among lower-income fourth graders and black fourth graders—scoring second and fifth in the nation, respectively.
To its vast credit, Mississippi has plainly been working very hard to improve its public school performance. We know of no reason—none at all—to think that anyone has misbehaved in any way in administration of the Naep.
That said, it has often turned out that impressive test scores aren't exactly what they seem. The Associated Press took those heartening data at face value.
In the past, this has often worked out poorly. Our question:
Should the AP have conducted a more detailed search?