Part 4—The liberal world’s rolling incompetence: The misstatements tend to be general when you read Diane Ravitch, who tends to be excessively partisan in support of whichever side she is currently on.
We assume that Ravitch is well intentioned. In certain ways, her new book, Reign of Error, is very useful, at least theoretically.
But Ravitch tends to overstate toward the goal of winning each partisan point. As an example, let’s consider something she said in a post this Tuesday.
The post was aimed at the D.C. schools, one of her favorite targets. She started with a statement which is semi-unknowable in principle, apparently false in fact:
RAVITCH (12/10/13): Despite its recent gains on the 2013 NAEP, the District of Columbia is not a national model.The post goes on from there, making various points about the D.C. schools. But why would Ravitch make the highlighted claim?
It remains the lowest performing urban district in the nation.
Does D.C. “remain the lowest performing urban district in the nation,” even after its ballyhooed score gains on the 2013 NAEP? Almost surely, it does not. Here’s a bit of background:
In one respect, the answer to that question is semi-unknowable. At present, the NAEP only reports stand-alone scores for about twenty urban districts. There are no such scores for dozens of the nation’s cities, including ten of the twenty biggest.
Full data for those twenty urban districts on the 2013 NAEP are not available yet. (D.C.’s scores have been released because it’s classified as a “state” for certain purposes.)
That said, it’s hard to believe that D.C. is the lowest performing urban district.
Because of its many charter schools; because it’s classified as both a city and as a “state,” with attendant confusion; D.C.’s data can be sliced and diced in many ways. But even on the 2011 NAEP, it’s hard to see how someone could say that D.C. was the lowest performing of the NAEP’s twenty urban districts. (Detroit’s scores were especially horrible.)
Given D.C.’s score gains in 2013, it becomes even less likely that it could be the lowest performing urban district.
Why did Ravitch say what she did? We have no idea, though it helps her drive some favorite points about D.C.’s pursuit of standard “reforms,” whose merits she was overstating just a few years ago.
With Ravitch, this tends to be what you get. That said, it isn’t just Ravitch.
Quite a few liberals have said odd things about the 2012 PISA scores, which were released last week. For our money, it was amazingly strange when Randi Weingarten, head of the AFT, said this in an instant press release:
WEINGARTEN (12/3/13): Today's PISA results drive home what has become abundantly clear: While the intentions may have been good, a decade of top-down, test-based schooling created by No Child Left Behind and Race to the Top—focused on hyper-testing students, sanctioning teachers and closing schools—has failed to improve the quality of American public education. Sadly, our nation has ignored the lessons from the high-performing nations. These countries deeply respect public education, work to ensure that teachers are well-prepared and well-supported, and provide students not just with standards but with tools to meet them—such as ensuring a robust curriculum, addressing equity issues so children with the most needs get the most resources, and increasing parental involvement. None of the top-tier countries, nor any of those that have made great leaps in student performance, like Poland and Germany, has a fixation on testing like the United States does.Whatever Weingarten may think of “top-down, test-based schooling,” her press release gives the impression that there has been no progress in American schools over the past decade.
From someone who represents the nation’s demonized teachers, that is a very strange post.
Have American schools “failed to improve” in the past decade? That isn’t what Weingarten explicitly said after her words are carefully parsed. But we’d say that impression is clearly conveyed by that remarkable post.
Have American schools “failed to improve?” That’s true if you only consider the PISA; if you assume its results can be trusted; and if you don’t disaggregate. And that’s what our new national cult—the cult of the PISA—wants us all to do!
The cult of the PISA worships two gods; their names are Andreas and PISA. As a result, this jealous new cult wants us to talk about scores from the PISA alone.
In her statement, Weingarten bowed to the wishes of this cult. But so did Ravitch, in her own post about the PISA scores.
As we noted yesterday, Ravitch drew four lessons at the end of her 1400-word post, My View of the PISA Scores. This was her very first lesson:
RAVITCH (12/3/13): From my vantage point as a historian, here is my takeaway from the PISA scores:That statement is simply astounding. In it, Ravitch seems to follow the edicts of the new cult of the PISA.
Lesson 1: If they mean anything at all, the PISA scores show the failure of the past dozen years of public policy in the United States. The billions invested in testing, test prep, and accountability have not raised test scores or our nation’s relative standing on the league tables. No Child Left Behind and Race to the Top are manifest failures at accomplishing their singular goal of higher test scores.
You can parse those words with care, as with Weingarten's statement. But those words convey a plain impression, just as Weingarten did.
Those words convey the plain impression that U.S. test scores have not improved in the past dozen years. And that claim is perfectly true—but only if you accept the edicts of the new PISA cult.
It’s true! There have been no gains in American scores—if you only consult the PISA, if you don’t disaggregate.
But what if you look at other tests, tests which may be more reliable than the somewhat unconventional PISA? What if you look at the TIMSS? What if you look at the NAEP?
On the NAEP, there have been tremendous score gains over the past dozen years. Her, you see one example:
Average NAEP scores, Grade 8 math, black studentsBy normal rules of interpretation, that looks like a very large gain. Bowing low to the cult of the PISA, Weingarten and Ravitch act as if those score gains don’t exist.
Since the rest of the “liberal” world doesn’t give a flying fig about what happens to American black kids, this ridiculous conduct by these “intellectual leaders” goes unnoticed, unremarked, unexplored, undiscussed. Because Ravitch is so influential, her ratification of the new cult spreads through the liberal world.
We have no earthly idea why Weingarten says the things she does. But please understand: to her credit, Diane Ravitch knows all about the NAEP!
In Chapter 5 of her new book, Reign of Error, Ravitch discusses the NAEP in detail, noting that she sat on its board of governors for seven years. (She was appointed by President Clinton.)
She goes on and on, sometimes overstating, about the primacy of the NAEP. This is the way the chapter begins:
RAVITCH (page 44): Critics have complained for many years that American students are not learning as much as they used to or that academic performance is flat. But neither of these complaints is accurate.“NAEP is central to any discussion of whether American students and the public schools they attend are doing well or badly,” she writes on page 45. Despite the usual puzzling blunders and the standard balls of confusion, she goes on to offer a detailed account of the major score gains on the NAEP, including score gains which occurred between 2000 and 2011 (see page 45).
We have only one authoritative measure of academic performance over time, and that is the National Assessment of Educational Progress, known as NAEP (pronounced “nape”).
It’s hard to know how we get from that detailed chapter to Ravitch’s blog post last week, in which she seemed to endorse the idea that no improvement has been recorded in the past dozen years. “No Child Left Behind and Race to the Top are manifest failures at accomplishing their singular goal of higher test scores,” an angry Ravitch wrote.
Carefully parsed, that claim may be true. But it conveys a plain impression, and Ravitch never mentions the NAEP or the TIMSS in her lengthy post.
Why did Ravitch write that unfortunate post about the new PISA scores? We’ll offer two possible guesses:
First, Ravitch is one of the most disorganized thinkers we’ve ever encountered in print. She constantly contradicts herself. (Compare her praise for the NAEP on page 263 of her book to her apparent wholesale condemnation of the NAEP exactly four paragraphs later.)
Second, she may have been unable to resist a partisan urge in last week's post—the urge to overstate against “testing, test prep, and accountability,” and against the two programs (No Child Left Behind and Race to the Top) which have embodied those approaches. When a partisan claim is there to be made, Ravitch rarely misses the chance to overstate it.
Whatever! Whatever her intentions may have been, Ravitch gave the impression in last week’s post that there has been no recent improvement. She never even mentioned the NAEP, or the many NAEP scores gains she details in her book.
Last week, it became quite clear that we have a new cult in this country.
That new cult is the cult of the PISA. Its adepts want you to do these things:
They want you to consult the PISA, and nothing else, when you assess academic performance.
They don’t want you to disaggregate scores. They want you to voice their gloomy conclusions about the lack of progress in our ratty schools, thanks to our ratty teachers.
This is exactly what Ravitch did in her peculiar post. Across the land, that gloomy impression spreads through the liberal world.
What have black kids ever done to have this unfaithful servant as their top liberal advocate? On the NAEP, they recorded that 20-point gain in math.
All across the United States, we liberals refuse to discuss it.
Tomorrow: How to keep pimping a comparison known to be bogus
A further note on PISA scores: For ourselves, we don’t have enormous confidence in the PISA.
That said, we’d like to see solid reporting about the different impressions one might gain from American scores on the PISA as opposed to American scores on the NAEP and the TIMSS.
Still and all, there have been some American gains on the PISA, especially after disaggregation. For example, here are the gains in science scores since 2006 (data for 2000 and 2003 are not available):
Average scores by American students, PISA science, 2006 versus 2012According to Amanda Ripley, 39 points is roughly comparable to one academic year on the PISA scale. On that basis, some of those score gains look fairly substantial, if you trust PISA samples.
White students: 523/528
Black students: 409/439
Hispanic students: 439/462
Asian-American students: 499/546
All American students: 489/497
We don’t have huge confidence in the PISA, but we’d like to see good solid reporting. Given the way our “press corps” works, that will of course never happen.
Instead, we’ll get what we got last week—the rise of our latest cult. Never mention those scores from the NAEP, its powerful adepts will tell us.