FOOLED ABOUT SCHOOLS: The latest scores!


Part 1—Keep gloom alive: In late November, the New York Times published an op-ed piece about our public schools.

The piece was written by Michael Brick, a former Times sports writer who has become an overnight expert on education. As Brick started his piece, he played a pair of familiar tunes:
BRICK (11/23/12): For the past three decades, one administration after another has sought to fix America's troubled schools by making them compete with one another. Mr. Obama has put up billions of dollars for his Race to the Top program, a federal sweepstakes where state educational systems are judged head-to-head largely on the basis of test scores. Even here in Texas, nobody's model for educational excellence, the state has long used complex algorithms to assign grades of Exemplary, Recognized, Acceptable or Unacceptable to its schools.

So far, such competition has achieved little more than re-segregation, long charter school waiting lists and the same anemic international rankings in science, math and literacy we've had for years.
Brick suggested—but did not say—that Texas’ schools are a bit of a mess. For liberals, this is the same pleasing tune Gail Collins sang in her massively uninformed new book, As Texas Goes.

That was Brick’s snark on the Texas schools. Concerning the nation’s public schools, Brick was more direct—though here too, he sang a familiar tune.

Concerning the nation’s public schools, Brick was no less gloomy. American schools produce “anemic international rankings,” he directly alleged. According to Brick, we’ve had these same anemic rankings “for years.”

Brick seemed to say that our scores are anemic—and that we’re making no progress. These are very familiar tunes. They’re sung all over the press corps.

At the time, we challenged Brick’s snark about Texas. We noted that, after disaggregation, Texas students score near the top of the nation on the National Assessment of Educational Progress, the widely-praised “gold standard” of educational testing.

We’ll guess that many readers were surprised by the strength of those Texas scores. Now, it’s time to review what Brick said about America’s schools.

Last week, the nation’s biggest newspapers reported the release of new international test scores. The new scores came from the TIMSS and the PIRLS, two of the three most widely reviewed international testing programs.

(All acronyms get unpacked below.)

Every four years, the TIMSS tests math and science in fourth and eighth grades. Every five years, the PIRLS tests fourth grade reading. Last week, these programs released their new international scores—and major newspapers in the U.S. reacted in two different ways.

Three of our biggest news orgs stuck to the familiar framework found in Brick's op-ed column. Below, you see the gloomy headlines those three news orgs produced:
Gloomy headlines about the new scores:
Associated Press, December 11: US students far from first in math, science
New York Times, December 11: U.S. Students Still Lag Globally In Math and Science, Tests Show
Washington Post, December 11: U.S. still trails Asia in student test scores
The gloom was general in those headlines. They told a very familiar tale—the same gloomy tale upon which Brick drew in his column.

In fairness, it should be said that those headlines are technically accurate. On the new international tests, American students actually are “far from first” in most areas (depending on what you mean by “far”). In that limited sense, our students do “lag globally in math and science,” just as that Times headline said.

Beyond that, it’s true that Americans students “still trail Asia” (their Asian counterparts) in most, though not all, test scores. That Post headline wasn't “wrong.”

When they reviewed these new test scores, the Post, the AP and the Times saw a familiar old story. But two other major news orgs seemed to see something different. This is what their headlines as said as they reviewed the new scores:
Upbeat headlines about the new scores:
USA Today, December 11: USA's schools move up in international rankings
Christian Science Monitor, December 11: How does US compare in math, science, reading?
Younger students do better
Two international studies show fourth- and eighth-grade scores in math, science, and reading in 2011. In the US, there's no cause for alarm, or celebration.
Our “younger students” are “doing better,” the Christian Science Monitor said in a large stack of headlines. And USA Today saw something similar. Its headline said that American schools have “moved up in international rankings.”

Please note: The cheerful headlines at those two papers don’t contradict the gloomy headlines from the Post and the Times. American schools could “move up in international rankings” (USA Today) and still “lag globally” (New York Times).

In our view, none of those headlines made a claim which was simply false. And yet, those dueling headlines almost seem to describe two different sets of test scores.

The headlines in the Post and the Times advanced a familiar old story. The headlines in the other two papers seemed to suggest something new. All this week, we will be looking at the actual test scores those headlines were describing.

We’ll also look at the full reporting done by those major news orgs.

As we review the actual scores, you will have to decide which of those news orgs seemed more perceptive in their appraisals. We will offer one criterion by which you might render your judgment.

Here’s the criterion: Will you be surprised by the data we show you? Will you find yourself saying this?

I had no idea!

Last month, readers may have been surprised to learn that Texas students score near the top of the nation, especially in math. If you were surprised by that fact, that may mean that you’ve been misled along the way—given a false impression.

As with Texas, so with the nation. We will guess that you may be surprised by some of these new international test scores. You may find yourself saying this:

How strange! I had no idea!

Your results may differ from ours. But in our view, the prophets of public school gloom and doom have been, on balance, misleading the public in recent years. In our view, USA Today’s Greg Toppo was more perceptive about these new test scores than was his counterpart at the script-ridden New York Times.

In our view, USA Today was more perceptive than the Post and the Times—more observant, more alert, more aware, more nuanced. Beyond that, we think the paper was simply more competent than the Times—but that is nothing new.

In our view, the Post and the Times are largely wed to a largely misleading old tale about public schools. In their news reports last week, they almost seemed to be trying to Keep Script Alive.

How gloomy should we be about our schools? If you’re surprised by the data we show you this week, you may decide that a gloomy old script has perhaps been misleading you too.

Tomorrow: Let’s take a look at the record!

The TIMSS, the PIRLS and the PISA: What the heck are the TIMSS and the PIRLS? Below, we unpack the acronyms of the three most widely reviewed testing programs:

The TIMSS: The TIMSS is more clumsily known as The Trends in International Mathematics and Science Study. Every four years, it tests fourth- and eighth-graders around the world in math and science. It last tested in 2011.

The PIRLS: The PIRLS is properly known as The Progress in International Reading Literacy Study. Every five years, it tests fourth-graders in reading. It last tested in 2011.

The PISA: The third major international program is the PISA—The Program for International Student Assessment. Every four years, it tests 15-year-olds in reading, math and science. It last tested in 2009.
Last week, the TIMSS and the PIRLS released new scores. Two major news orgs saw positive signs. Three other news orgs tilted toward gloom.

Which news orgs were more perceptive? All week, you get to decide!


  1. Should newspaers always provide a narrative? In this case, different organs reported different interpretations for the new test results. But, why do they have to assign meaning at all? Couldn't they just present the results along with a comparison of the past and let the readers decide what the data tells us?

  2. Newspapers nedn't *always* provide meaning, but most often they should. It needn't be a 'narrative,' but the role of the journalist is not just to repeat information but to help the consumer interpret that information.

  3. Small correction. PISA tests every three years and it cycles making reading math and science the major topic. Reading was the major topic in 2009 and math was the major topic in the 2012 round. Results from 2012 will be reported in the fall of 2013. PISA had been managed by the Australian Educational Research Council since its inception in 2000. Starting with the 2015 cycle (science will be the major topic) PISA will be managed by Educational Testing Service.

  4. Too sleepy this morning to even get the name of the people who contract with me right. It is the Australian Council for Educational Research.

  5. ETS, now headed by Dan Coleman?