MONDAY, JUNE 26, 2023
...concerning one basic point: We disagree with (the long-admired) Kevin Drum with respect to one basic point.
Before we revisit that basic point, let's recall why Nicholas Kristof claims that a revolution is underway in Mississippi's public schools.
Simply put, Mississippi's Grade 4 reading scores can look extremely good! For a full rundown, click here for last Thursday's report.
Those reading scores can look very good. Here are three examples of what we're talking about:
Average scores, Grade 4 reading
Lower-income kids, 2022 Naep
U.S. public schools: 202.67
Mississippi: 211.74
Average scores, Grade 4 reading
Lower-income black kids, 2022 Naep
U.S. public schools: 193.42
Mississippi: 202.76
Average scores, Grade 4 reading
Lower-income white kids, 2022 Naep
U.S. public schools: 211.49
Mississippi: 224.45
For all Naep data, start here.
Holy cow! On the 2022 Naep reading test, Mississippi's lower income Grade 4 students outscored their counterparts across the U.S. by almost one academic year!
Lower-income black kids also recorded a gap of that size when compared to their peers nationwide. Mississippi's lower income white kids were on the good end of a gap which was even larger.
On their face, those numbers can look extremely good. Something else is true, of course—an experienced, savvy observer won't automatically put his faith in such numbers.
It isn't that people cheat on the Naep. As far as we know, there's virtually no way to do so. Also, there are few incentives for anyone to do so, except perhaps on the level of a state superintendent.
That said, there are many ways for testing programs to produce misleading data. Any experienced person will automatically understand that—but through no particular fault of his own, Nicholas Kristof isn't an experienced education analyst.
Based upon his June 1 essay in the New York Times, Kristof pretty much accepted those Grade 4 reading scores at face value. In his essay, he said those apparently impressive scores were part of "a revolution" in Mississippi, a revolution based on new, improved ways of employing "the science of reading."
Having said that, how (perhaps slightly) odd! As you can see, Mississippi's Grade 4 scores on the Naep math test were almost equally impressive;
Average scores, Grade 4 math
Lower-income kids, 2022 Naep
National public schools: 222.73
Mississippi: 229.13
Average scores, Grade 4 math
Lower-income black kids, 2022 Naep
National public schools: 212.16
Mississippi: 218.80
Average scores, Grade 4 math
Lower-income white kids, 2022 Naep
National public schools: 232.50
Mississippi: 241.74
Holy cow all over again! As it turns out, Mississippi's lower-income Grade 4 kids outscored their counterparts in math by margins which were also quite impressive.
No one claims that Mississippi has found some new, improved way to teach grade school math. But the state's large array of lower-income kids outscored their counterparts nationwide in Grade 4 math as well, by a margin roughly two-thirds the size of their margin in reading.
Friend, are you inclined to think that data like those should be taken at face value? Long ago, we ourselves might have reacted that way—but long experience has taught us that (pleasing) data like these can be misleading, even where all parties are behaving in total good faith.
Example:
Way back in 2006, the Washington Post ran a report across the top of its front page about the outstanding test scores at little, low-income Maury Elementary in Alexandria, Virginia. The banner headline said this: A Study in Pride, Progress.
After examining the statistical fine print, we showed that Maury's third graders actually had the second lowest reading scores of any school in the whole state of Virginia! In that case, the gross misdirection tracked back to several ludicrous administrative and statistical procedures which had been engineered within the state's education department.
For our interview with the president of the Virginia Department of Education, you can just click here. Everyone agreed that the Post had been grossly misled by Mury's reported scores, but so were Post readers, of course. In our own direct and varied experience, this sort of thing had already been going on for decades at that point.
There are many ways for test score data to be significantly misleading. However pleasing such test scores may seem to be, data like those from Mississippi actually can be misleading.
So can drive-by claims of "miracles" or "revolutions" from inexperienced journalists—from well-intentioned journalists who spend a day or two visiting some high-scoring school or school system, all the while believing the various things they get told.
If we actually care about the lives of this nation's low-income kids, we'll be very careful about the conclusions we draw from such pleasing data. Truth to tell, our tribunes in the press corps rarely exhibit such care, and our "education experts" simply never speak up when the state of play in the pubic schools is being misdescribed.
Question:
If no one cheated in any way to produce Mississippi's Grade 4 scores, how could those impressive scores possibly be misleading?
You're asking a very good question! Luckily, we explained the possible problem with Mississippi's Naep scores way back on June 5, in this somewhat detailed report.
We explained the possible statistical problem in detail that day. Here's the way the long-admired Drum has now (quite cogently) thumbnailed that possible statistical problem:
DRUM (6/23/23): Bob is skeptical of these scores for another reason: Mississippi's reforms included something called the "third-grade gate," which means holding back kids who can't pass a reading test at the end of the year. This is obviously going to improve scores for 4th graders, but it's a bit of a statistical mirage.
On the other hand, only 9% of Mississippi's third-graders are held back, so the effect is probably modest.
It's just as Kevin says! As we explained on June 5, holding back an unusual number of lower-performing third-graders will almost surely "improve [the state of Mississippi's average] scores for 4th graders."
Also this:
To the extent that Mississippi's scores improve for that reason, the improvement in Grade 4 scores is, in fact, "a statistical mirage," just as Kevin says.
Most simply put, a state which makes a lot of kids repeat third grade will almost surely have a statistical advantage over other states which don't. It's hard to make a valid comparison between states which hold a lot of kids back as opposed to states which don't.
In that passage, Kevin is describing the statistical problem we detailed on June 5. Having said that, we don't necessarily agree with his claim that the statistical advantage gained by Mississippi will "probably [be] modest."
As we showed you on June 5, the lowest-performing ten percent of fourth graders produce very low scores on Grade 4 Naep tests. Conceivably, eliminating those lowest performers from the population which gets tested could improve a state's average score by a fair amount, especially in the first year of the third grade retention program.
Does Mississippi's Grade 3 retention program explain those high Grade 4 Naep scores? We'll answer your question in this time-honored way:
Goofus doesn't want to bothered with such boring, tedious questions. Gallant insists that we should check the possibility out.
Putting it a different way, we would tell you this:
People who care about low-income kids won't be inclined to accept such feel-good test score data at face value. That said, alas:
For the past fifty years, our education journalists, such as they are, have tended to peddle the "feel good" stories, and they have then walked away. As for our alleged "education experts," they have tended to slumber and snore every darn step of the way:
Those alleged education experts! They slept through the endless cheating scandals which we first described, in the Baltimore Sun, back in the 1970s. Decades later, it fell to newspapers like USA Today to report the astonishing cheating scandals our alleged education experts had agreed to ignore.
Also this:
As we endlessly noted at this site, Naep scores improved for all demographic groups, on a nationwide basis, all through the 1990s and the 2000s—but so what? Even as those "disaggregated" Naep scores continued to climb for all major groups, our education experts refused to inform the public of this important fact, even as our hapless mainstream journalists kept insisting that "nothing has worked" in our public schools.
It's hard to have sufficient contempt for the decades of silence from these expert lambs. That said, this is the way our national discourse actually works, even among the academics and journalists who comprise our blue tribe's honored elite.
Our discourse works this way with respect to the public schools, but also with respect to a wide array of other subject areas. When it comes to the public schools, our education experts refuse to speak up. So too with massively-paid "cable news" stars like Rachel, Chris and Lawrence. You'll never hear about Mississippi's kids on our tribe's cable news shows.
In short, our blue tribe's elites are, at best, just extremely careless. At their worst, they don't seem to care about Mississippi's low-income kids, and the odds say they never will.
We disagree with Kevin Drum about the possible size of the "statistical mirage" created by that third-grade retention program. Before we liberals pleasure ourselves by saying a "revolution" is underway, we need to spend a bit more time pondering those somewhat unlikely statistics.
Is a revolution really underway in Mississippi's schools? Has a "Mississippi miracle" taken place in recent years?
We've seen this kind of careless journalistic behavior many times over the past fifty years. Our journalists feed us a pleasing tale about some vast success somewhere. As a general matter, we don't discuss our low-income schools for the next quite a few years.
In our next report, we'll show you what Kristof's revolution looks like on the Grade 8 level. Spoiler alert:
On the Grade 8 level in Mississippi, Kristof's "revolution" looks like a revolution which has failed. Or did that alleged revolution simply never happen? Our blue tribe's history is quite clear here. It has been a very long time since we actually seemed to care.
Mississippi is full of good decent (low income) kids with good decent (low income) parents. As Langston Hughes once put it, life for them ain't been no crystal stair.
On the highest levels of our own tribe's discourse, no one actually cares about such children and such parents. Over the course of the past fifty years, the evidence in support of that claim has been surprising but clear:
Within the elites of our self-impressed tribe, no one cares about low-income kids. Also, the odds are quite good that nobody ever will.
Why don't our tribe's major stars discuss the lives of Mississippi's kids? For those who are willing to see, the answer is blindingly clear.
Tomorrow or Wednesday: Mississippi's Grade 8 scores
After that: The expert's tale
Phonics.
ReplyDeleteWow. Can anyone imagine the hell it would be to run into Bobby at a cocktail party? A good editor could have condensed this and all other recent musings on this subject into a few small sentences, no more than 4-5 paragraphs. Because there is so little substance. A few assumptions here, some explicitly stated. Now I could take Bobby's approach and meander through a Sargasso sea of verbiage to arrive at those points, and you, the reader, would, nearly drowning in it, be left exhausted. Does Bobby have the retention rates for the comparative programs outside of Mississippi? Maybe he does and will reserve disclosure until sometime in August, when this diatribe finally runs its course. Does he have anything but conjecture about how the retained students did among their new classmates. The few retained students I went through grade school with remained near the bottom among their new classmates. Some surely had attention deficit, others fetal alcohol syndrome, others on the spectrum, some outliers with respect to age(sees Gladwell) and others with priorities that do not get fixed by an additional year. With Bobby and his argument it's all gonna be conjecture because of the dirth of information he has to throw up in a couple of tables. There is nothing to sort through. A good statistician could make a few assumptions and come up with how well that 9% would have to do in their retained grades to impact the scores, but I am not him. But that said, I take the liberty to be skeptical of Bobby's meager premise.
ReplyDeletedirth—> dearth
DeleteOne suspects there is SOMETHING to be said for Bob 's argument here, as there always was when he challenged basic assumptions about Clinton/Gore. But his utter lack of effective style has kept him ineffectual, As his poor mental patient says, "for better or worse."
DeleteYou dudes all need to get a life. What, do you wait up all morning for Bob’s posts? You getting paid for this or something?
ReplyDeleteLook who's talking?
DeleteI don’t spend 500 words going off about him at 8 a.m.
DeleteYou get the difference? Get a life, folks. You might too want to spend a bit of time reading him as well. You know, for accuracy’s sake?
“Dudes”, hilarious!
DeleteIf Somerby has an actual point, he first needs to deal with the study posted by mh, that concluded that the retention did not produce the improvement noted on the NAEP. Somerby today continues to ignore that analysis.
ReplyDeleteOne cannot argue that because there was cheating on other high-stakes tests in the past, that these scores in MS, based on considerable work and change in the reading instruction, must FIRST be considered the result of cheating on a cheat-proof test, before anyone can acknowledge that hard work has paid off for many of the students in MS.
Today's essay illustrates only that Somerby is a very stubborn person and will not admit it when he is mistaken.
"One cannot argue that because there was cheating on other high-stakes tests in the past, that these scores in MS, based on considerable work and change in the reading instruction, must FIRST be considered the result of cheating on a cheat-proof test"
DeleteGood. No one has.
Somerby has.
DeleteI myself posted excerpts from several newsletters within MS discussing controversies with implementing their improvements in reading instruction. These describes in detail the situation before they began their improvements in state instruction, the need for teacher training, how it was introduced (in stages), and how much money was allocated for schools to do this work.
ReplyDeleteSomerby has never addressed that issue -- if there are major changes made and a great deal of money and effort directed toward schools over a ten year period, why wouldn't there be improvement in how well kids can read?
If MS began doing what other districts do nationwide and did that work across a decade, why wouldn't their reading scores come up to the level of those other schools? That is what happened. Somerby has shown nothing unique about the MS kids that would have prevented them from learning, just as kids (yes, even poor and black and Hispanic kids) do in other states.
Charges of cheating are very serious. Somerby has presented no evidence of cheating at all in MS. He has even agreed that the NAEP test is difficult to cheat on, and that no one cares enough about the results to cheat on that test. Why then is he once again insisting that cheating must be the first explanation when a school does well. And he seems to be saying that one must explore and set aside the idea of cheating before accepting the results. No one does that, for good reason. The situations with cheating are a very small subset of all the districts who take the NAEP. There is no reason to suspect fraud. AND THERE IS NO FUCKING EVIDENCE THAT THE GOOD DECENT KIDS OR THEIR TEACHERS AND ADMINISTRATORS IN MS CHEATED ON THE NAEP.
This post does not make an accusation of cheating. It raises concerns about potential statistical issues and misleading data.
DeleteReading can be fun!!
Somerby does too talk about suspecting cheating as the first reaction to good news about schools. Go back and read his essay.
DeleteThe post ("does too") mention cheating but does not make accusations of cheating as was falsely claimed.
DeleteTalking about the need to eliminate cheating as an explanation of improvement is an implied accusation of cheating, especially on the cheat-proof NAEP test.
DeleteImplied accusations are 100% subjective.
DeleteWhy bring up cheating at all if Somerby is not intending people to think MS cheated in some way? It would be irrelevant, as it in fact IS.
DeleteThis is like saying that Tiger Woods won the Master's tournament, but before congratulating him one must consider the possibility that he cheated, even though no one has ever been accused of cheating at that tournament, and it would be very difficult to do, but it is theoretically possible because someone else somewhere else was once found to have cheated at golf. And then, one must also consider that his scorecard may contain some statistical anomalies, even though it was verified when he turned it in and no irregularities were found. And then calling all those sportscasters who applauded Tiger's win careless and lazy because they didn't check for cheating and statistical mistakes, meanwhile forgetting to applaud Tiger for all the hard work he has put into his long career, especially coming back after injuries and surgeries.
This is what Somerby has done here over the past month, except substitute the children and teachers of MS for Tiger Woods and the NAEP test for the Master's tournament.
And now some idiot troll thinks it is funny to pretend Somerby didn't say what everyone has seen him say here.
That has the be one of the most unconvincing and absurd analogies ever written. Congratulations for that.
DeleteThe trolls here are the best evidence that Somerby is a creature of the right. Why would they defend a has-been liberal expounding on esoteric details of NAEP testing if they didn’t have their own marching orders?
Delete7:25
DeleteCheating was brought up for a few different reasons. First to establish that cheating was not a reason for the increase in test scores. Second, to show how education experts and the press have tendency to overlook flaws in the system and not take stories of this nature critically. He's just warning us not to believe everything we read and take everything at face value. It's pretty simple. Yeah and that analogy man. Woah.
That is completely classic!
DeleteIdiot Troll: Bob accused Mississippi of cheating!
A. No, he didn't.
Idiot Troll: Well it was an implied accusation!
A. That doesn't matter.
Idiot Troll (crying): Well why did he bring up cheating then!!!????
A. To specifically state cheating wasn't the reason
Idiot Troll: Troll.
Did anyone bring up cheating when Tiger won The Masters, just to say he hadn’t cheated? How that have been received?
DeleteWas it a lie or a mistake when Tiger won The Master’s? Why would skepticism be needed? Somerby presented no plausible evidence that the MS test results were unreliable. All of his speculations were addressed in comments. They were as empty as someone saying that Tiger cheated. Someone trying so hard to discredit Tiger’s victory might be considered a racist. I feel the same way about Somerby’s effort to discredit the MS schools.
DeleteGreat trolling as usual. You are the undisputed king!!
DeleteTiger is a jerk.
DeleteBob’s not accusing anyone of cheating.
DeleteIndeed, notably not Trump.
Delete"... it is theoretically possible because someone else somewhere else was once found to have cheated at golf."
DeleteThat someone was the 45th President of the United States of America, except he didn't only cheat once.
Neither Somerby nor Drum is an expert in education or testing.
ReplyDeleteDrum is a generalist.
DeleteHis degree is in journalism.
DeleteThey are though both very excellent critical readers.
DeleteDrum worked in marketing.
DeleteSomerby and Drum are both dim witted neoliberals.
Both Somerby and Drum read at grade level.
Delete3rd Grade level.
DeleteAnd they can do basic math.
DeletePerhaps this persistent attack on MS reading instruction arises from a different source. Does Somerby even want kids to learn to read better. Previously, he has defended book banning, especially Maus.
ReplyDeleteToday Tom Sullivan (at Digby's blog) describes the use of Hitler quotes by right wingers focused on controlling education:
"Hitler resurfaced again over the weekend at the Faith & Freedom Coalition’s “Road to Majority Policy Conference” at the Washington Hilton. Presumably, the context was the triple threat of critical race theory, “woke,” and “grooming” in schools.
“Basically, this is an evangelistic movement on the left…. It’s indoctrination. They are proselytizing to the next generation,” alleges Liberty University’s Ryan Helfenbein in a Right Side Broadcasting Network (RSBN) interview.
“What we’re discovering as parents and conservatives is — wait a second — education really is evangelism. So, if you don’t control education, you cannot control the future. And Stalin knew that, Mao knew that, Hitler knew that. We have to get that back for conservative values.”
Nobody indoctrinates our kids but us MAGA authoritarians, dammit! (Tom Sullivan at Hullabaloo says)
https://digbysblog.net/2023/06/26/hitler-resurfaces-at-faith-freedom-coalition-conference/
We assume that because Somerby was himself a teacher that he automatically wants kids to read better, that his sympathies are with education and the goals of public education in arming citizens to participate in a democracy. But maybe that is not what he wants.
If you want to control the hearts and minds of children, then teaching them to read and giving them access to a variety of viewpoints is the last thing you want. Undermining the efforts of MS teachers to equip black students with better reading skills is a step toward the kind of voter control and suppression that MAGAts seek, especially in areas where there are blue voters, such as among Southern black families.
I see Somerby's month+ long crusade against progress in MS as an attack on an institution that benefits democracy and Democratic voters by undermining faith in the institution of public education. Today he calls them cheaters. How better to convince parents that the schools are indoctrinating students, not teaching them to read?
Maus was never banned.
DeleteSomerby criticized the use of the word "book banning", suggesting it was an exaggerated characterization that led to an unproductive pseudo-discussion.
Somerby supported the banning on the grounds that parents should be allowed to have books removed. His sophistry over the word banning was unconvincing because the book was no longer available and he was ignoring the other places in the country where that was happening as part of a right wing effort, not parent concern.
DeleteThank you for providing your interesting subjective interpretation of what was written.
DeleteDenying that Somerby said what he said is just another form of trolling.
DeleteThen the commenter here is trolling by saying Somerby supported book banning. Maybe they feel like that was also "implied."
DeleteHe’s not calling anyone cheaters, and how is keeping people back a grade a sign of “success”?
DeleteIt’s kind of cooking the books, isn’t it?
You do realize that that Faith assembly was not quoting Hitler approvingly?
DeleteYou guys do realize that, don’t you?
The Liberty U guy saying they need to do the same as Hitler, the Moms of Liberty quoting Hitler, the Moms coughing to interrupt a moment of silence for Holocaust victims; in aggregate, there is, at minimum, an implied affinity for Hitler, to some degree, in some aspect.
DeleteEven being as charitable as warranted, it’s sickening to consider the traumatizing abuse these people suffered as children, for them to reach this level of hatred and bitterness.
Quoting Hitler as a way to expose a supposed leftist agenda does not compute, as it’s trivially ahistorical. Hitler was a right wing fascist, just as Liberty U and the Moms are. Hitler’s early victims were leftists and gays, the cohorts these right wing fascist organizations are after as well.
I doubt we will see concentration camps any time soon, but these right wingers are bent on making the lives of non right wingers a living hell, while endeavoring to continue the generational cycle of child abuse that they themselves suffered through.
Classic internet troll response. Ignore the issue at hand, attack the messenger by implying they love Hitler based on the fact that an unrelated group quoted Hitler.
DeleteLol, brilliant. You win the internet today!
In a related twist of irony, amgry troll shows below grade reading comprehension in their inability to discern if the article they are attacking called the kids cheaters. Lol, classic stuff. I cant read very well, but i can compare you to HITLER!!
2:14 are you ok? Are you having a stroke?
Delete“Hitler” is a foreign name, but it can be successfully read with English phonics.
DeleteHistorians have uncovered substantial evidence showing that Hitler was a Nazi.
DeleteHigher test scores are God’s reward to Mississippi for accepting His servant Tate Reeves as governor.
ReplyDeleteMaybe the threat of being left back produced a real improvement, by motivating students and their parents to work harder.
ReplyDeletePeople who struggle with reading are not lazy. The have a cognitive deficit, their mind works differently so they need different instruction, which is why reading specialists, assessment, and individual attention is needed.
DeleteDavid, why don’t we all speculate, just like our hero Bob, without doing a lick of investigation. Sounds like par for the course.
DeleteActually hearing the tape that might put Trump away really underlines the dogged, stupid stubbornness that Bob has appoached these awful years with. Bob's long held foolishness about not being able to prove what Trump was up to is is shattered into a million pieces. Yes, Trump is nuts, but what of people who thought so little of the US that they would make excuses for someone putting the very foundation of the Nation in jeopardy? Is it really just some old blindness about the poor, victimized South? We'll never know, but how someone (allowing that yes, Bob might have been some kind of legacy) could get through maybe our most prestigious University and still be a hapless dolt? How much did Bob ever put his own reading skills to use?
ReplyDeleteExactly what does that Trump tape say? That he did or did not show that document to people?
DeleteThat he did, and laughed about it, as if it were amusing to share stolen classified documents.
DeleteWeird guy, that Trump. Pervert, too.
The only way the tape hurts Trump politically, is if he called for Reparations for black people on it.
DeleteNo expert, but reading is the key to all learning. Expect that better reading would allow MS students to do the maths more better too.
ReplyDeleteReading is a key, but there are other vital keys for learning, and other ways of learning that do not involve reading.
DeleteReading is the key to much learning.
DeleteVladimir Kogan says the latest NAEP scores are very bad.
ReplyDeletehttps://www.the74million.org/article/new-naep-scores-reveal-the-failure-of-pandemic-academic-recovery-efforts/
The same publication has an article showing 13 year old (long term) NAEP scores in math and reading, and those scores do not match the catastrophizing exhibited in Kogan’s article.
DeleteThe scores are down a little from their peaks, but they have been on a downward trend since 2012.
Furthermore, kids from earlier times with lower scores did not bring about the ruin of our society.
Higher scores do not lead to what Kogan is hoping for, which he makes evident at the end of his article:
“Recent high school graduates and those who will graduate over the next several years are almost certain to be the least prepared to enter the labor market or college in several generations.”
Or, to paraphrase, let’s get those scores up so we can generate better wage slaves!
In reality, the 74 is a pro charter schools, anti union right wing organization funded by right wing stalwarts like the Walton’s and Betsy DeVos.
Democratic Party is abandoning Biden.
ReplyDeleteIt's too bad Mayor Pete has performed so poorly. At one point I thought we could have tapped him. But it might have to be Mr Guilfoyle.
Delete5:08,
DeleteBecause he brought the internet to rural areas?
No this is a broad section of Democrats from rural and u.. rban areas. The poll did not ask them to give a specific reason but vast majority were very unhappy with Scranton Joe Who knows why exactly. He's not even able to give a press conference. But you know how we all felt after he dropped to minimum wage and the public option healthcare in order to focus on a war across the other side of the world. Personally, I still support him.
DeleteThey have no idea why they are unhappy with Biden. It could be the way he tamed inflation. It could be the infrastructure programs. it could be the lowest unemployment rate in over half a century. it could be they like don't like the spending on Ukraine, and they'd rather that money stay in the United States , where Republicans would do everything in their power to assure it never makes it in to the hands of the needy.
DeleteOne things for sure, if they can't put it into words, the Right-wing corporate media will do it for them.