VISIONS OF PISA: Ravitch sells a bogus claim!


Part 5—The return of the Age of Aquarius: Liberal response to the new PISA scores has often been quite hard to fathom.

Most amazing is the acceptance of the new cult of the PISA. This involves the apparent acceptance of PISA scores as the sole measure of student achievement.

In response to weak aggregate scores on the PISA, major liberals have failed to mention the large score gains which have been recorded on the NAEP, our widely-praised domestic testing program.

We’ve failed to mention American scores on the TIMSS and the PIRLS, the major international tests on which American students score better than on the PISA.

It’s astounding to see major liberals behave that way—to see our team advance the idea that American schools have recorded no progress in the past dozen years. But this is hardly the limit of our tribe’s incoherence and cluelessness—or of the apparent dishonesty of one of our loftiest leaders.

For our money, many of Diane Ravitch’s reactions to the PISA have been rather foolish. For one more example, consider this post, in which she links to a post by D.F. Brandenburg and showers him with praise.

Brandenburg is a former math teacher in the D.C. schools. At his eponymous blog, he has done a lot of good work about various public school issues.

We thought his reactions to the new PISA scores were a bit overwrought and unwise, to the extent that we could understand what he was proposing.

Ravitch linked to Branderburg’s seven-point post about the PISA, praising him for his “words of wisdom.” In the passage shown below, Ravitch quotes the part of his post she admired most.

To Ravitch, what follows is brilliant, unassailable. We found ourselves flashing back to the 60s:
RAVITCH (12/5/13): And here is his most brilliant, unforgettable, unassailable point:

“Arne Duncan and his ilk say that the fact that the same approach has failed for 10 straight years, means we need to keep doing it harder. Sensible people would say no, let’s forget about measuring with stupid standardized tests. Let the kids learn, remember that humans LOVE to learn stuff—it’s what we do as a species. And precisely nobody knows what knowledge of today is going to be the most useful or fun tomorrow. So let’s get rid of the idiotic focus on standardized tests and Big Data, and stop wasting so much money and time and energy on them. We’ve got all sorts of art and sports and drama and dance and music and technology and building stuff and real science and history and psychology to learn and to perform.”
That was the end of Ravitch’s post. We had the analysts check our papaya juice to see if someone had slipped us some acid, even on a weekday.

It’s certainly true that standardized tests have been misused and abused in a wide array of ways. For our money, it would be unwise to “forget about measuring with stupid standardized tests” as a result.

In the past, in the absence of such external checks, the majority culture tended to ignore the lagging performance of our so-called minority kids. Parents of minority children were persistently lied to about the progress of their own individual children. On a wider scale, the public was lied to too.

Results from an annual standardized test make that much harder to do.

We think it would be very unwise to eliminate standardized tests—but in our view, the second part of that highlighted statement is even stranger.

Back at the dawning of the Age of Aquarius, we heard these same ideas. “Let the kids learn. Remember that humans LOVE to learn stuff—it’s what we do as a species.”

Let’s get the teachers out of the way and let the children learn!

In fairness, there’s a very strong germ of truth lurking inside that bell-bottomed sentiment. Good schools will often help children discover the joy of the search—the pleasure of reading; the fun of learning and analyzing; the importance of outreach and service.

That said, children won’t necessarily take these route on their own. That is especially true of children who may be years behind traditional norms in basic reading and math skills.

We’re sorry, but it isn’t the case that those beautiful kids will magically learn how to read on their own. For our money, Branderburg’s statement is off in the clouds, perhaps in the sky with diamonds.

That said, Ravitch’s new book, Reign of Error, is full of similar sentiments. In this passage from her post about the new PISA scores, she links similar airy-fairy ideas to a dose of American exceptionalism:
RAVITCH (12/3/13): The more we focus on tests, the more we kill creativity, ingenuity, and the ability to think differently. Students who think differently get lower scores. The more we focus on tests, the more we reward conformity and compliance, getting the right answer.

Thirty years ago, a federal report called “A Nation at Risk” warned that we were in desperate trouble because of the poor academic performance of our students. The report was written by a distinguished commission, appointed by the Secretary of Education. The commission pointed to those dreadful international test scores and complained that “on 19 academic tests American students were never first or second and, in comparison with other industrialized nations, were last seven times.” With such terrible outcomes, the commission said, “the educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a Nation and a people.” Yet we are still here, apparently the world’s most dominant economy. Go figure.

Despite having been proved wrong for the past half century, the Bad News Industry is in full cry, armed with the PISA scores, expressing alarm, fright, fear, and warnings of imminent economic decline and collapse.

Never do they explain how it was possible for the U.S. to score so poorly on international tests again and again over the past half century and yet still emerge as the world’s leading economy, with the world’s most vibrant culture, and a highly productive workforce.
“Students who think differently get lower scores?” So do kids who can’t read.

As every flower child once knew, only conformists score well on tests! On balance, that statement is silly. And in that full passage, Ravitch comes dangerously close to accepting the mediocre American scores of the past—scores which were largely built on the very weak performance of deserving minority kids who were floundering badly in school and were badly in need of help.

Was that old regime really OK? Again and again in her current writings, Ravitch comes dangerously close to suggesting that only the squares would worry about the meaning of the low average scores this nation produced in the past.

We know where those low scores came from. We don’t think it was cool or OK.

According to Ravitch, “we are still here, apparently the world’s most dominant economy.” It’s amazing to see the liberal world mixing this strand of exceptionalism with the flower-child foolishness of the past. But Ravitch persistently pushes these themes as she thrashes about, trying to denounce the “reforms” on whose behalf she was pushing the overstatements just a few years ago.

In our view, Ravitch is strongly drawn to overstatement, whichever side she happens to be on at whatever time. Consider this post from December 5, in which she continues to promote a comparison which she knows to be bogus.

Ravitch’s post, which seems rather slick, starts with these words:

“Daniel Wydo, a teacher in North Carolina, sent this analysis of 2012 PISA.”

She then presents a 935-word essay in which Wydo makes a series of misleading comparisons between American PISA scores and those from higher-scoring nations.

In her current book, Ravitch notes that comparisons of this type are “technically” bogus. In her December 5 post, she made those comparisons anyway, speaking through Wydo’s words.

Midway through her post, Ravitch is still quoting Wydo. She lets him make a stirring claim she surely knows to be bogus:
RAVITCH (12/5/13): “This is not a new phenomenon. For every administration of PISA and TIMSS, when controlling for poverty, U.S. public school students are not only competitive, they downright lead the world.”
Even in context, that statement by Wydo is so imprecise that it’s virtually meaningless. That said, it conveys an upbeat but thoroughly bogus impression.

No, Virginia and Oklahoma! Even after “controlling for poverty,” American students do not “downright lead the world” on international tests in any meaningful sense.

In that post, Ravitch uses Wydo’s words as a way to promote a set of bogus comparisons and claims. In his essay, Wydo makes a type of comparison which is now quite popular among liberals who have been misinformed by figures like Ravitch in the recent past.

It’s popular, but it’s thoroughly bogus. Here’s what Wydo has done:

First, he takes the average PISA scores produced by American schools in which fewer than ten percent of students qualify for free or reduced price lunch. Then, he compares the average scores produced by those schools to the average scores produced by entire foreign countries.

Here’s what’s wrong with that:

For starters, “free or reduced price lunch” is not a measure of poverty. Wydo doesn’t seem to know that.

At present, roughly half of American students qualify for free or reduced price lunch. As a result, very few American schools fit the category Wydo describes—and these schools tend to be found in our most affluent neighborhoods.

In his comparison, Wydo isn’t presenting the average scores for the full range of American students “after controlling for poverty.” Instead, he is presenting the average scores for a small slice of the student population, a slice of students who attend schools in our most upscale neighborhoods.

What percentage of our student population attends these schools? What is their average socioeconomic rank? There’s no way to know such things! For that reason, it makes no sense to compare their average scores to the average scores of entire nations.

(Nor does that slice of our population always “lead the world.” On the 2012 PISA, those American students were outscored in math by the entire student populations of Singapore, Hong Kong, Taiwan and Korea. Their average score was still quite good. But almost surely, they are a small, highly upscale slice of the population.)

In the last few years, these bogus comparisons have become quite popular among liberals. But they’re useless, bogus, grossly misleading—and to some extent, Ravitch knows this. In her new book, she continues to present these comparisons. But she includes this disclaimer (page 65):

“Technically, the comparison is not valid, because it involves comparing students in low-poverty schools with the average score for entire nations.”

That disclaimer grossly understates the problem with this type of comparison. But so what? Responding to the new PISA scores, Ravitch dispensed with disclaimers altogether. She simply posted what Wydo had sent.

It was Wydo who said it, not her!

It ought to be an embarrassment to see Ravitch argue this way. It ought to be, but it isn’t. Our “liberal” world more closely resembles the world of Fox with each passing day.

Can we talk? In the laziness of this unchallenged work; in our refusal to do any better; in our insouciance about these topics; in all these ways, we proudly display our liberal contempt for the interests of black kids. We also display the broken state of the American discourse.

Why should children know how to read and cipher? Our intellectual leaders can’t!

Tomorrow—Epilogue: The cult of the PISA! What Zakaria said


  1. Is Somerby saying that the PISA test is responsible for rising scores of minorities over the last 30 years and we ought to keep participating in it for this reason?

    Ravitch is on record as praising the NAEP test because it is not a high stakes test, and it is used for diagnostic purposes (which is what tests ought to be used for) and not to purposely and with malice aforethought to stigmatize and punish teachers as recommended by the rightwing "reformers". But don't look over there, look at the liberals.They are sometimes sloppy and hyperbolic (and also slumber in their sleep).

  2. THANK YOU commenter

    WTF somersby, are you saying (in your mumble-mouth, passive-agressive EXTREMELY repetitive and tiresome way), that kids and parents are NOW armed with info from this testing regime that allows them (!?) to improve ?
    what utter horseshit...
    the VAST majority of the kids WE ARE FAILING don't give a shit, and neither do their parents, IF THEY EVEN KNOW what their 'scores' are... IF they even have any meaning...
    (Not to mention, approximately 99.99% of the time, the kids/parents are NOT given any timely info about these test scores, it is ALL AFTER THE FACT...)

  3. Superb series of posts, simply superb, Bob.

  4. Neither kids nor parents, nor teachers are given results of these tests, they are being used to "rate" (read fail) schools, and teachers, which is not what they were designed for. Furthermore, some of the "value-added" scores (originally developed to be used in talking about agricultural products) have an error rate of 30 percent. But don't pay any attention to that, it is all the fault of the latte drinking, Hampton-weekending liberals, according to Somerby.

  5. I agree with the "superb series of posts" anonymous.
    As for the "WTF" anonymous, are you calling for no standardized testing? How would that radical idea work?

  6. There is no standardized testing in Finland, which has scores comparable to Massachusetts, or higher.

    Testing ought to stay in the classroom with the results a private matter between teacher and pupil, except perhaps, for a college entrance exam.

  7. Here we go again. Somerby is reproached for not mentioning Ravitch's objection to the testing regime generally, and he uses the occasion not to reflect, but to further promote his narrative. Age of Acquarius indeed!

    The issue is poverty, Bob. It doesn't take a clairvoyant, 30 years in the classroom or a Ph.D in Education to recognize that poor children coming from stressed homes don't do well in school. Period. The educational fads come and go, but kids in stable surroundings learn even in mediocre schools and with medicore teaching. The test scores themselves bear this out: students with similar backgrounds score similarly, across national boundaries. High achieving average performance on average may well reflect better teaching or particular educational goals. But so what? These differences aren't disabling, can be surmounted in higher education even if they amount to a full year or more of difference and don't predict economic performance.

    In other words, the testing regime is nonsense. If schools are woefully underperforming -- kids can't read, etc. -- the education system (and administering tests) can do little or nothing, because the education system isn't the cause.

    1. Yes, poverty is a serious issue.

      The corporate-style "reformers" argue (wrongly) that making teachers and schools "accountable" (through test scores) will correct the poverty problems and restore American economic competitiveness.

      As the late Jerry Bracey used to say, "And pigs will fly."

      Meanwhile, the corporate "reformers" embrace and lobby hard for economic policies that perpetuate economic inequality and send huge taxpayer subsidies to Wall Street, big bankers and hedge funders.

      If we invest in promoting the general welfare of the citizenry – and that means alleviating poverty – then we can expect to see even more improvement in the quality of public schools.

  8. Has anyone questioned the science in equating tests across so many languages? It would seem to be an impossible task to measure high-order thinking skills so that scores on tests in English equate precisely with test scores in Chinese or any of the other non-English speaking PISA-taking entities. Yet we assume without question that this feat has been accomplished validly and reliably.

  9. "First, he takes the average PISA scores produced by American schools in which fewer than ten percent of students qualify for free or reduced price lunch. Then, he compares the average scores produced by those schools to the average scores produced by entire foreign countries. "

    That's because the top tier of U.S. students and schools (with less than 10% of students qualifying for free/reduced lunch) is comparable, in and of itself, to many other countries in whole. For instance, based on OECD standards, Finland enjoys a child poverty rate hovering around 5%, while the U.S. approaches 25%. Our top tier in SES is comparable to Finland in totality. You must travel way down Finland's totem pole of poverty before reaching kids that are comparable to U.S. kids based on SES.

    You obviously don't understand.

    "For starters, “free or reduced price lunch” is not a measure of poverty. Wydo doesn’t seem to know that."

    Really. This is hilarious. And you don't even need me to point out how stupid this is, because you completely contradict yourself here:

    "At present, roughly half of American students qualify for free or reduced price lunch. As a result, very few American schools fit the category Wydo describes—and these schools tend to be found in our most affluent neighborhoods."

    Question Einstein - if free/reduced lunch rates are not a measure of child poverty, then why do the schools with less than 10% of students qualifying for free/reduced lunch "tend to be found in our most affluent neighborhood." If free/reduced lunch rates are not a measure of child poverty, then a-duh, why are ones with less than 10% of kids qualifying for free/reduced lunch found in our more affluent neighborhoods.

    Anyways, the feds stipulate free/reduced lunch rates specifically by definition. Do you know what it takes to qualify? It doesn't sound like you do.

    "What percentage of our student population attends these schools? What is their average socioeconomic rank? There’s no way to know such things! "

    Oh my. If they don't qualify for free/reduced lunch, especially on the level of whole school, we know that most of these kids are not poor.

    I really feel bad for you. I really do.

  10. with a Pandora Sale low heel) Pandora Jewelry sale i've been Cheap Pandora Jewelry searching all over and Pandora friendship Charms have had no luck, Pandora Sister Charms this size chart Pandora Animal Charms is dead Pandora Retired Charms on,is_customized Pandora Number Charms ,August 2,shows Pandora Letter Charms off every herve leger unwanted lump herve leger sale and even herve leger outlet supermodels struggle herve leger dress sale to carry cheap herve leger it off herve dress A duplicate herve leger dress outlet of every herve leger replica outfit,Mike Sorrentino ' here worst tenant ever' according Pandora Mum Charms to landlord Pandora Wedding Charms seeking to Pandora heart Charms evict Jersey Shore star click here from rental Pandora friendship Charms home New Pandora Sister Charms legal troubles Pandora Animal Charms No wonder Pandora Number Charms she's happy,Pandora Letter Charms com/cgi

  11. Chi Flat Iron Black Friday day off. Coach Handbags And Louis Vuitton Outlet unlike most Cheap Oakley Sunglasses celebrities, Coach Outlet her Coach Purses chicest moments happen off the red carpet.Typically, we Cheap Oakley Sunglasses reserve plaid for our Michael Kors Outlet Online casual days, but a set of street style gurus Toms Shoes Outlet Online have made Chi Flat Iron us rethink the Red Bottom Shoes staple.Michael Kors Factory Outlet From '50s Coach Outlet silhouettes to glam metallics, see Ray-Ban Sunglasseshow they give the grunge trend an elegant new spin. You know how sometimes Chanel Outlet you look at your closet-which Coach Outlet is overflowing to Toms Shoes Outlet Online the point Toms Shoes where Coach Factory Outlet you have Ray Ban Outlet

  12. Coach Outlet Store Online shirts all living on Cheap Ray Ban the same Coach Outlet hanger Michael Kors Outlet and it takes ALL of your core Coach Handbags strength to Beats By Dre move things Cheap Oakley Sunglasses around-and Oakley Outlet you wonder Coach Outlet whether you Cheap Oakleys should (a) Valentino Rockstud weed through it to give some stuff away, (b)send a True Religion Outlet shipment to live at Coach Factory Outlet your mom's house, or (c) just move? Co Co Chanel This is one of Oakley Outlet those times. Louis Vuitton Outlet Stores Kim Kardashian showed Michael Kors off her Christian Louboutin Shoes curves in a Oakley Sunglasses Oakley Outlet Store Online Hollister Outlet jumpsuit on Ray-Ban Aviator Sunglasses Wednesday Coach Outlet when she Michael Kors Outlet touched Abercrombie Outlet down at Los Ray Ban Sunglasses Angeles Valentino Shoes International Airport.