Ninth letter tells the tale: On Sunday, December 22, the New York Times published a set of nine letters about our public schools.
The first eight letters all seemed to endorse a familiar claim—a familiar claim we would regard as grossly misleading. The fourth letter stated the gloomy premise most clearly:
The stagnant results of the international PISA exam have spoken: An extensive overhaul in the American education system is desperately needed.An extensive overhaul is desperately needed! In our view, that familiar claim tends to misrepresent the results from last year's Pisa tests. For our previous reports about those letters, click here.
That said, the assessment stated in that letter is familiar and very gloomy. By now, almost everyone has internalized that assessment. Everyone is inclined to recite it, as those first eight letters show.
Those first eight letters advanced a gloomy assessment—a gloomy assessment which had emerged from the a highly selective Times report on the newly-released Pisa scores. Those first eight letters recited sweetly. It's the soul of our journalism and of our floundering discourse.
The ninth letter did something which is almost never done. It blew the whistle on a gong-show column—a column which had appeared in the Times on December 6.
In that column, Emily Hanover had delivered an upbeat report about rising test scores in Mississippi.
Hanover is an education reporter for American Public Media. Mississippi's Grade 4 reading scores have been on the rise since 2013, she excitedly wrote in the Times. Indeed, fourth graders in our poorest state were now meeting the national average in reading, she said.
Why were those Naep scores on the rise? Mississippi began stressing phonics in 2013, Hanover somewhat implausibly wrote. That said, she chose slightly loftier language. The headlines on her column say this:
There Is a Right Way to Teach Reading, and Mississippi Knows ItThe state was relying on cognitive science! Based in part on an earlier colun in the Times, it was clear that she meant the teaching of phonics.
The state’s reliance on cognitive science explains why.
Mississippi's reading scores were on the rise because the state had started teaching phonics! As we noted yesterday, this didn't quite seem to make sense. Mississippi's scores had risen by a similar amount in Grade 4 math. After disaggregating the scores by race, the scores seemed implausibly high, given the fact that Mississippi is our poorest state.
Could those rising scores, and that high achievement, really have come from something as simple, and as basic, as the teaching pf phonics? To us, the claim didn't exactly seem to make sense—and then, the ninth letter appeared!
The letter came from "a retired education professor at Hunter College." It posited a different reason for the rise in those Naep scores:
LETTER TO THE NEW YORK TIMES (12/22/19): Emily Hanford’s piece about improved test scores in reading in Mississippi since the state began funding in 2013 to train its teachers in a particular methodology certainly sounds optimistic. However, there’s another reason, a big one, for the improvement in fourth-grade reading scores, which Ms. Hanford didn’t mention.Say what? According to the retired professor, that "particular methodology" (the teaching of phonics) didn't provide the sole explanation for Mississippi's improved reading scores.
In 2013, Mississippi passed a Literacy Based Promotion Act, which mandated that in most cases, a student scoring at the lowest achievement level on the state-mandated third-grade achievement test won’t be promoted to fourth grade. Voilà! The weakest readers in third grade don’t move up to fourth grade, and the fourth-grade reading scores go up.
She said there was another reason for the improvement—"a big one." Mississippi has started making its weakest readers repeat third grade, the retired professor said.
The weakest readers in third grade don’t get promoted to fourth. As a result, the average scores in fourth grade goes up!
Could a change in promotion policies explain the rise in Mississippi's scores? And remember—the scores have also risen in Grade 4 math. It isn't just reading scores!
We checked what the professor had said. The New York Times should be embarrassed by what we found.
In fact, Mississippi did adopt a new promotion policy back in 2013. The state did begin requiring more of its lower-performing students to spend an additional year in Grade 3 before moving on to fourth grade.
Eventually, those kids do take the Grade 4 Naep—unless they get "promoted" directly to fifth grade, a type of trick which was used in Texas during the Bush and post-Bush years. Assuming they doend up in fourth grade, they get an extra year to develop their reading and math skills before they take the Grade 4 Naep. Presumably, this tilts the field in Mississippi's favor on those Grade 4 tests.
Does this explain Mississippi's rising scores in Grade 4 reading and Grade 4 math? Does this explain why Mississippi's black and white fourth graders are both outscoring their national peers in both reading and math, by a substantial margin?
AS far as we know, there's no way to measure the extent to which this retention policy has affected the state's Naep scores. But the situation had been widely discussed long before Hanford wrote her column, and it's astonishing that she would have written her column for the Times without discussing this matter.
As noted, Hanford's column appeared in the Times on December 6. Eight months earlier, the Associated Press had offered this full-length report about the toughened retention policy. Along the way, the AP's Jeff Amy included this note:
AMY (4/20/19): The mandatory retention policy remains controversial nationwide. Experts agree students who flunk a grade are more likely to drop out. While third-grade reading policies typically call for intensive remedial work for students who are held back, one study found the boost helps for a while but eventually fades.In theory, literacy coaches are a good thing. In theory, so is extra money for summer school. So is additional training on the teaching of reading, though all good ideas can be bungled.
The Republican policymakers who adopted Mississippi's plan from Florida support it, pointing to improvements in performance on a nationwide test. Mississippi is paying for literacy coaches to help improve instruction in 182 of 420 schools statewide with a third grade. The state has also provided training on teaching reading to 13,000 people, and provides extra money for summer schools for struggling readers.
For ourselves, we're inclined to think that making students repeat one or more grades does more harm than good. Others take a different view. As far as we know, there is no Official Established Truth concerning this question.
That said, if State A is retaining a lot of third graders and State B isn't, that would tend to tilt the playing field on the Grade 4 Naep in State A's favor. And uh-oh! According to Amy, Mississippi makes a lot of kids repeat a grade long before Grade 3:
AMY: Mississippi has long flunked the largest proportion of young students nationwide, often students from poor households who enroll lacking groundwork for academics. Last year, Mississippi held back 9% of kindergartners, 8% of first graders and 6% of second graders.We can't tell you if that's a constructive policy. It does mean that a lot of kids taking the Grade 4 tests in reading and math are actually fifth- or sixth-graders by age, with one or two more years of instruction than other fourth grade kids.
(Maddeningly, Amy didn't say how many of the state's third graders had been held back last year. For that number, see below.)
In the first passage we quoted, Amy said that Mississippi policymakers were "pointing to improvements in performance on a nationwide test." Presumably, that was a reference to the Naep—and sure enough:
Five weeks before Hanover's column appeared in the Times, The Hechinger Report had offered a full-length report about Mississippi's rising scores. The report had appeared beneath this double headline:
Mississippi made the biggest leap in national test scores this year. Is this controversial law the reason why?Hechinger describes itself as "an independent, nonprofit newsroom" which focuses on education issues. In her report, its reporter, Bracey Harris, directly referred to the Naep:
Education officials credit efforts an increased focus on childhood literacy, including a controversial retention policy, for the academic gains.
HARRIS (11/1/19): Mississippi is one of 16 states requiring students to show a certain level of reading ability in order to pass third grade, although some research suggests that holding students back can hurt rather than help them. More than 3,300 Mississippi students, or about 10 percent of last year’s third graders, have to repeat third grade this year, more than double the number of third graders retained the year before, according to the state Department of Education.The potential effects of the state's retention policies had long been under discussion. It's amazing to think that Hanover never even mentioned this matter in her upbeat, somewhat implausible column for the perpetually hornswoggled Times.
State officials may have reason to believe that raising the bar will pay off. Between 2011 and 2017, Mississippi’s fourth graders posted the second-largest improvement on the NAEP reading exam in the country. Over the past four years, the percentage of fourth, fifth and seventh grade students performing at a proficient or advanced level on the state’s English language arts exam has also increased.
That said, the story gets worse! In her somewhat implausible column, Hanover pompously attributed Mississippi's score gains to its embrace of "the science of reading." In its headline, the Times jacked that formulation up, referring to "cognitive science."
Based in part on an earlier column, it seems clear that this was a reference to the teaching of phonics. But where did Hanover get that somewhat pretentious language?
Ugh! On December 4, yet another report—this time by the Fordham Institute—offered the horrible highlighted point. Again, we're including the headline:
COLLINS (12/4/19): Mississippi rising? A partial explanation for its NAEP improvement is that it holds students backSomeone had even "adjusted for demographics." Imagine doing that!
One of the bright spots in an otherwise dreary 2019 NAEP report is Mississippi. A long-time cellar dweller in the NAEP rankings, Mississippi students have risen faster than anyone since 2013, particularly for fourth graders. In fourth grade reading results, Mississippi boosted its ranking from forty-ninth in 2013 to twenty-ninth in 2019; in math, they zoomed from fiftieth to twenty-third. Adjusted for demographics, Mississippi now ranks near the top in fourth grade reading and math according to the Urban Institute’s America’s Gradebook report.
So how have they done it? Education commentators have pointed to several possible causes: roll-out of early literacy programs and professional development (Cowen & Forte), faithful implementation of Common Core standards (Petrilli), and focus on the “science of reading” (State Superintendent Carey Wright).
At any rate, Hanover had simply adopted the language of the state superintendent when she offered her pompous point. The state had focused on "the science of reading." Forget all those kids it held back!
To what extent do Mississippi's scores result from its retention policies? We know of no way to say.
Will those policies work out well in the end, or will they simply increase the ultimate drop-out rate? We can't answer that either.
We can tell you this:
It was virtual journalistic malpractice when Hanover failed to mention this factor in her rather implausible column for the Times. Meanwhile, regarding the Times, what more can we say?
Especially in the realm of public education, if you read it in the New York Times, you should start checking your wallets. The paper's work in this field has been awful forever. It's skillful at getting it wrong.
For ourselves, Hanover's column never exactly seemed to make sense. Nor did her work seem especially competent. Her failure to disaggregate scores—to adjust for demographics—struck us as a typical manifestation. In the realm of public schools, our discourse runs on such fuel.
We checked one way Naep scores can be gamed, at least in theory. If a state disqualifies an unusual number of students from taking the tests on the grounds of disability, that state can tip the scales in its favor on the Naep.
A few states did this a few years back; in response, the Naep tightened its procedures. When we checked the Naep's data on this point, we found that Mississippi's disqualification rates have been very low, as is now the norm across the nation.
That said, widespread grade retention can likely tilt a state's average Naep scores. It was virtual malpractice when Hanover blew right past this point, even as she kissed the keister of the state superintendent's explanation for the success of his own scientific approach.
Needless to say, this bungled column appeared in the Times. If it concerns the public schools, and it appears in the New York Times, you can safely assume that something is wrong.
Feckless incompetence of this type leads to a Trump.
Tomorrow: The only "solution" that counts