Part 4—King of the cherry-pickers: Like all public schools which ever existed, American public schools could do a better job.
On the elementary level, they could do a better job transmitting the basic skills of reading and math. They could do a better job conveying the spirit and the enjoyment of inquiry—that so-called “love of learning.”
With kids who are struggling to meet conventional norms, our schools could do a better job helping children “catch up.” They could do a better job challenging kids who may be especially “talented.”
That said, how good or how bad are American schools? In recent years, how good a job have these schools been doing?
In a nation as large and as varied as ours, that’s a tough question to answer. Last Saturday, in the New York Times, Jal Mehta pretended to try.
In an unusually lengthy op-ed column, Mehta told a very conventional story. We’d say his story was false.
Mehta is a young assistant professor at Harvard, where quite a few students got booted this year in a slightly strange case involving academic integrity.
How different the standards seem to be for Harvard’s assistant professors! In his lengthy column, Mehta copied off the papers of a wide range of our so-called “educational experts.” He cherry-picked his basic facts within an inch of their lives.
This is the portrait Mehta drew at the start of his very lengthy column. The portrait he drew is very familiar, but we would be inclined to call this portrait baldly dishonest:
MEHTA (4/13/13): In April 1983, a federal commission warned in a famous report, “A Nation at Risk,” that American education was a “rising tide of mediocrity.” The alarm it sounded about declining competitiveness touched off a tidal wave of reforms: state standards, charter schools, alternative teacher-certification programs, more money, more test-based “accountability” and, since 2001, two big federal programs, No Child Left Behind and Race to the Top.Mehta started with A Nation at Risk, as people of his class always do. From there, he drew a gloomy portrait of “stubborn mediocrity,” of an ongoing lack of progress.
But while there have been pockets of improvement, particularly among children in elementary school, America’s overall performance in K-12 education remains stubbornly mediocre.
In 2009, the Program for International Student Assessment, which compares student performance across advanced industrialized countries, ranked American 15-year-olds 14th in reading, 17th in science and 25th in math—trailing their counterparts in Belgium, Estonia and Poland. One-third of entering college students need remedial education. Huge gaps by race and class persist: the average black high school senior’s reading scores on the National Assessment of Educational Progress continue to be at the level of the average white eighth grader’s. Seventeen-year-olds score the same in reading as they did in 1971.
As the education scholar Charles M. Payne of the University of Chicago has put it: “So much reform, so little change.”
People like Mehta can, and do, tell this gloomy tale in their sleep. In order to do so, they deliberately cherry-pick their facts, even as they pray for the day when American teachers will be just a bit more like them.
In that passage, Mehta tells a very familiar tale. In the process, he deceives the American public in a very familiar way. The Times should never have published this crap. But this highly scripted, familiar old crap gets peddled somewhere every day.
Here’s what’s wrong with that portrait—and by wrong, we mean “grossly misleading.” Let’s talk about Mehta’s international data, then about his domestic test scores:
International comparisons: At present, the United States and other developed nations take part in three major international test batteries—the PISA, the TIMSS and the PIRLS. American students tend to score better on the TIMSS and the PIRLS, so people like Mehta will frequently stick to the PISA alone.
In that passage, Mehta cites U.S. performance on the 2009 PISA, fails to mention U.S. performance on the 2011 TIMSS and the 2011 PIRLS. American students scored reasonably well on those latter two batteries.
Perhaps for that reason, those more recent tests disappeared.
Some scholars think the PISA is a superior test battery to the TIMSS and the PIRLS; this may be Mehta’s opinion. (Other scholars take a different view.) But the time has come to stop the process by which readers are only told about the tests which produce the lowest scores.
For information regarding the 2011 testing, see our notes below. Let’s add one point before we move on the domestic data, where Mehta’s conduct is simply disgraceful:
In a very typical manifestation, Mehta plays the flag-waving chauvinist, seeming to sniff at the very idea that American students could “trail their counterparts in Belgium, Estonia and Poland.”
That passage ought to be embarrassing. Consider Belgium alone.
Belgium isn’t an underdeveloped Third World nation, struggling with the effects of past colonialism. Belgium is a developed, advanced western European nation. It was one of the colonizers!
There is no reason why we should assume that American students will, on average, be more accomplished in reading and math than students in Belgium are. For one thing, American schools face certain challenges based upon our brutal racial history, challenges which are not faced by Belgian schools. Whatever their other failings, the Belgians didn’t spend three hundred years trying to eliminate literacy among a large swath of their population.
Our benighted ancestors did. The tragic effects are still apparent in our public schools.
Different countries face different types of demographic challenges; it’s hard to balance the equities when we discuss such matters. At any rate, Belgian students did outscore American students in reading on the 2009 PISA, but only by a narrow margin. (“Statistically insignificant,” the PISA documents say.) In math, they outscored American students by a wider margin.
On the 2011 TIMSS, Belgian students narrowly outscored American students in math—but American students outscored their Belgian peers by a wider margin in science. And American students smoked the Belgians, by a wide margin, on the 2011 PIRLS. (The PIRLS tests reading alone.)
Having said that, forget what we’ve told you! If you read the New York Times, you will only hear a mocking complaint about the way the Belgians outscore us. The snark comes straight from the planets Jingo and Ugly American. And as Mehta snarks, the mixed outcomes in the data are simply withheld.
Should it be shocking if U.S. students are outscored on the PISA by students in tiny Estonia, a literate Baltic state? (In reading, U.S. students were outscored by one point, 501-500. Estonia did not take part in the TIMSS or the PIRLS.) Actually no, it should not be. But people like Mehta will play every card as they tell the gloomy tale that is loved in one part of our world.
The domestic data: Mehta’s international cherry-picking ought to be an embarrassment. It should be an embarrassment for Harvard and for the New York Times.
On the domestic front, his conduct is a disgrace. Let’s consider the various ways he cherry-picks from the National Assessment of Educational Progress (the NAEP), the widely-praised “gold standard” of domestic educational testing.
How does Mehta cherry-pick? Let us count the ways:
He cherry-picks data on the basis of age. He cherry-picks data by dealing in average scores alone—by failing to “disaggregate.”
To some extent, he cherry-picks data by citing the NAEP’s Long Term Trends Assessment, rather than the parallel study which is typically called the Main NEAP. He cherry-picks data by citing “gaps by race and class” without explaining that the gaps persist in spite of the large score gains shown by black and Hispanic students.
You see, white students have shown substantial score gains too. These score gains have worked to maintain the gaps, although the gaps have grown smaller. But those “gaps by race” persist despite large gains by black and Hispanic students.
Would you have any idea of such facts from reading Mehta’s account?
“So much reform, so little change?” Since Mehta himself has chosen the NAEP; since he himself has chosen to cite the NAEP’s Long-Term Trend study; since he himself chose to measure progress from 1983; since Mehta himself has made those selections, let’s see what kinds of change has occurred on the Long-Term NAEP since that ballyhooed date.
Warning: In what follows, we aren’t trying to tell you why the score gains in question occurred. Did they happen because of better instruction? Did they happen because of the removal of lead from the air?
We aren’t trying to answer those questions. We’re simply showing you some of the data our bright fellow disappeared.
The data which disappeared: Poor Mehta! He’s saddened by the mediocrity he can see all around him! He’s saddened by the lack of progress—by the “failed profession” of teaching.
He’s saddened to see that American teachers don't have a “professional code” to match those of other major professions—presumably, to match the lofty professional code observed by scholars like him.
“So much reform, so little change,” one of his experts exclaims. In this way, New York Times readers—American citizens—are once again being deceived.
What kind of change has occurred on the Long Term NAEP since 1983? That’s the testing program Mehta chose, and the baseline year.
Thanks for asking! This kind of change has occurred since that time and since 1971, when the NAEP program began:
The Long Term NAEP tests 9-year-old students, 13-year old students, and 17-year-old students. Because drop-out rates have decreased over time, it can be a bit difficult to construct meaningful comparisons among 17-year-olds.
But here’s what has happened among 9-year-olds and 13-year-olds, where almost everyone always was in school. We’ll measure from 1983 and from the early 1970s, when NAEP testing began. We’ll adjust for one minor hiccup in testing procedure the NAEP made along the way:
What kind of “progress” has been displayed by 9-year-old black students in reading? Good God! From 1971 to 2008, the average score for this group increased by 37 points, an astonishing gain on the NAEP scale. From 1984 to 2008, the gain was 21 points. (2008 is the most recent year for which Long Term NAEP data are available.)
Those are very large score gains. A rough rule of thumb is often used when journalists discuss the NAEP. According to this rule of thumb, ten points on the NAEP scale is roughly equal to one academic year.
(Warning: Journalists only apply this rule of thumb when it produces gloomy conclusions. That is, they apply it to the achievement gaps, not to the score gains of each racial group, which typically go unreported.)
Do ten points really equal one academic year? We regard that as a very rough rule of thumb. But it helps you see how crazy it is when Mehta selects the Long Term NAEP, then moans about “so little change.”
Those were the score gains in reading. What kind of progress has been displayed by 9-year-old black students in math?
Good question! From 1973 through 2008, the gain has been that same 37 points. (1973 was the first year the NAEP administered math tests.)
Among 13-year-old black students, the gains have been similar, both since the start of the NAEP and since A Nation at Risk. In reading, the average score for this group rose by 30 points from 1971 through 2008. In math, the average score rose by 39 points from 1973 through 2008.
Taken on their face, these are very large score gains. How can a scholar cite the NAEP, then moan about “so little change?”
He can do so by cherry-picking, the skill at which Mehta excels.
How much progress has occurred on the NAEP? Consider one remarkable fact from the so-called Main NAEP, the companion program to the Long Term Trends Assessment.
Surely, Mehta has heard this fact. We’ll quote educational expert Richard Rothstein, citing this remarkable fact in Slate:
ROTHSTEIN (8/29/11): The only consistent data on student achievement come from a federal sample, the National Assessment of Educational Progress. Though you would never know it from the state of public alarm about education, the numbers show that regular public school performance has skyrocketed in the last two decades to the point that, for example, black elementary school students now have better math skills than whites had only 20 years ago.Astonishing! As of 2007, black fourth graders were scoring higher in math than white fourth graders scored in 1992. That is an astonishing fact. No one could have foreseen it.
Surely, Mehta has heard that fact. It's hard to know how a scholar can consider that fact from the program which he himself cites, then moan about stubborn mediocrity and “so little change.”
Tomorrow, we’ll continue this sad rumination. Mehta will still be holding his job along the banks of the Charles.
Tomorrow: Quite reminiscent of Pravda
Scores on the TIMSS and the PIRLS: How did American students score on the 2011 TIMSS and the 2011 PIRLS? What countries did they surpass?
See our award-winning series, Fooled About Schools. For part 1, just click here.
THE POT CALLING THE KETTLE BLACKReplyDelete
"Mehta is a young assistant professor at Harvard, where quite a few students got booted this year in a slightly strange case involving academic integrity."
"How different the standards seem to be for Harvard's assistant professors."
Somerby provides no evidence that Mehta was involved in any way in the "slightly strange case involving academic integrity."
Instead he puts two facts - Mehta is an assistant professor at Harvard and there was a cheating scandal at Harvard - together in a single sentence to imply that Mehta was involved in the scandal and, therefore,that Mehta is a hypocrite.
This is cherry picking at its most dishonest, all to paint Mehta as a dishonest, cherry-picking hypocrite. It is also illustrative of the First Rule of Somerby Discourse: do as Somerby says and not as he does.
Reading comprehension is dead. Maybe Mehta could opine on that.Delete
This is a bullshit comment. There wasn't the slightest implication that Mehta was involved; it was a used as a commentary on Harvard. Without being able to associate himself with Harvard, Mehta never would have had an op-ed published in the NYT. It's legit in this context to raise questions about the academic integrity of Harvard. We shouldn't give this guy anymore respect because he is from Harvard. Indeed, as with most of the like-minded "reformers" who had the benefit of very pricey private schools and Ivy League colleges, and never in their lives had to touch a public school classroom with a 10-foot pole, he likely knows less about public education in this country than someone with actual classroom experience in public schools teaching at Podunk State Teachers College.Delete
Sorry, but this TDH post isn't cherry-picking. It is grounded completely in fact and sound analysis. I have been the first to accuse TDH of cherry-picking when (not infrequently) he has done it -- such as focusing two comments before this one on a single inartfully written sentence by Mehta on a legitimate point of analysis, the greater proliferation of education schools in the U.S. compared to Finland. However, in this case, Mehta's op-ed is exactly what TDH says it is. It is cherry-picked in the extreme from the various international and national tests, in the effort by Mehta to align himself with the prevailing (and false) standard narrative of failing public schools. I would say it's a transparent attempt by Mehta, with the New York Times helping the launch of that attempt, to become the go-to guy on public education for the Times, the Washington Post and other media outlets who either want to promote the standard narrative regardless of the evidence or at minimum want to make sure it has a hearing. No IT manager ever got fired for choosing IBM; likewise for any editor who picks a guy from Harvard.
This is an excellent analysis. THIS is something that ought to appear in the New York Times Magazine, but the editor there is guaranteed not to have the guts to run it.
"Mehta is a young assistant professor at Harvard, where quite a few students got booted this year in a slightly strange case involving academic integrity."Delete
Sorry, urban, but this is the same rhetorical tricks Dubya used to link Saddam to 9/11 without having the guts to come right out and say it.
Somerby implies that the dishonesty shown in Mehta's article is comparable to the lack of integrity shown by the students -- he is not implying that Mehta helped any students cheat.ReplyDelete
You don't understand the term cherry-picking -- it refers to picking only the data that selects your point and ignoring disconfirmatory data. It does not refer to making false analogies or false claims.
Mehta is dishonest in the way he provides support for his assertions. Somerby has made his case and it can be verified by anyone who goes to the NAEP data independently. Professors are trained in graduate school to evaluate data properly. Mehta is either poorly trained or being deliberately misleading. Since he was hired by Harvard, it seems unlikely he was poorly trained.
Professors at Harvard have done dishonest things to advance their careers. Look at the recent case of Marc Hauser. Harvard may place unique pressures on their new professors to attain high visibility as scholars. That encourages cheating among Professors, just as competitiveness and unduly high expectations place pressures on students to cheat.
This website was... how do I say it? Relevant!ReplyDelete
! Finally I have found something which helped me. Appreciate it!
Feel free to visit my web-site Visit Website
Hello, I enjoy reading through your post. I like to write a little comment to supportReplyDelete
my web site - Converse Pas Cher
These are really wonderful ideas in regarding blogging.ReplyDelete
You have touched some good points here. Any way keep up wrinting.
Feel free to visit my website: Michael Kors
Excellent weblog here! Additionally your site lots up fast!ReplyDelete
What host are you the use of? Can I get your affiliate hyperlink
in your host? I want my site loaded up as quickly as yours lol
My web page: extra resources
What's up, I read your blogs like every week. Your humoristic style is witty, keep it up!ReplyDelete
my website ... Read More Here
I think this is one of the most significant information forReplyDelete
me. And i'm glad reading your article. But wanna remark on some general things, The website style is wonderful, the articles is really nice : D. Good job, cheers
my web blog; Nike Air Max
Established transportation infrasctructure for a plethora of lifestlye,ReplyDelete
dining and entertainment.the interlace condo