We return to that New York Times news report!

SATURDAY, NOVEMBER 2, 2013

Motoko Rich, functionally illiterate: You’re right!

This past week, we never got back to that news report in the New York Times about international test scores.

The report appeared on October 24, atop the front page of the National section. It was written by education reporter Motoko Rich.

For our first post on this subject, click here.

Our question was this: How is it possible that someone like Rich could write such an utterly clueless report? Given our focus on Minnesota’s test scores, we never got back to that topic in our posts this week.

If you want to puzzle about your nation’s journalistic culture, we’ll suggest that you peruse that report. Focus on three topics:

Clueless from the headline down: Rich’s report concerned a new study of U.S. math and science scores on the 2011 TIMSS. We marveled at the headlines on the hard-copy report, and at its opening paragraph:

RICH (10/24/13): Better News In New Study That Assesses U.S. Students/
Majority Outperform International Average

Amid growing alarm over the slipping international competitiveness of American students, a report comparing math and science test scores of eighth graders in individual states to those in other countries has found that a majority outperformed the international average.

The headlines mirror Rich’s opening paragraph, although they may miss one nuance. That said, those headlines are quite hapless. Here’s why:

From those headlines, a reader would think it came as “news” to learn that a majority of U.S. students outperformed the international average on the tests in question.

In fact, TIMSS results from 2011 were released last December. Since that time, it has been known that American students outperformed the international average in math and science at both grade levels tested.

On the TIMSS scale, the international average is set at 500. These were the United States’ average scores:

Average scores, United States, 2011 TIMSS
Grade 4 math: 545
Grade 8 math: 509

Grade 4 science: 544
Grade 8 science: 525

Grade 8 math was the weakest area for American students. But everyone has known since last December that American students outscored the international average on all four TIMSS tests.

Everyone has known that except the New York Times’ readers! At the Times, Rich wrote a confusing report last December about the new TIMSS scores. In this latest report, she and the Times seemed to think it was “news” that U.S. students had exceeded the international average.

(In her opening paragraph, Rich may mean that “a majority” of states “outperformed the international average.” Given the average scores for the nation as a whole, it's hard to know why even that would be surprising.)

It’s hard to get more clueless than this. But Rich and her unnamed editors were equal to the task.

Reinforcing a ubiquitous bogus script: Let’s return to that opening paragraph. In her very first sentence, Rich reinforces a bogus, ubiquitous script:

RICH: Amid growing alarm over the slipping international competitiveness of American students, a report comparing math and science test scores of eighth graders in individual states to those in other countries has found that a majority outperformed the international average.

Has the international competitiveness of American students been “slipping?” That is, of course, a ubiquitous claim. But is this famous claim true?

As far as we know, there is no international forum in which scores by American students have been “slipping.” (Or “deteriorating,” the harsher term Rich employs a bit later on.)

American test scores could surely be better. And many folk do express alarm about the idea that those scores are “slipping.”

But in fact, American scores have not been “slipping” or “deteriorating.” Unless you read the New York Times, a famous newspaper which seems determined to keep confusion alive.

Can’t stick to the question at hand: As she continues, Rich described the findings of the study on which she was reporting. Near the end of her report, she returned to the claim that U.S. scores have been “deteriorating” in recent years.

In this passage, Rich seems to examine that claim. She seems to examine that claim but, in various ways, she fails. The journalistic work in this passage is just astoundingly bad:

RICH: A growing chorus of critics have said that claims of deteriorating American student performance have been exaggerated. In her recent best seller, “Reign of Error,” the education historian Diane Ravitch wrote, “Contrary to the loud complaints from the reform chorus, American students are doing quite well in comparison to those of other advanced nations.”

But others say the international comparisons are apt. Paul E. Peterson, the director of the program on education policy and governance at Harvard and a fellow at the conservative-leaning Hoover Institution, said the study provided less good news than people might think.

He pointed out that the pool of countries taking the math and science exams included many developing countries, and that several industrialized nations, including France, Germany and Denmark, did not participate. “So if you really want to compare the U.S. to the developing world, then we do look good,” Mr. Peterson said.

Truly, that is awful. Start with the first paragraph:

“A growing chorus of critics have said that claims of deteriorating American student performance have been exaggerated,” Rich writes. To many, this presentation will convey the idea that student performance has been deteriorating. It’s just that the degree of deterioration has been exaggerated.

In fact, there has been no “deterioration” at all. A competent journalist would simply say that.

As she continues, Rich quotes a statement by Diane Ravitch. This statement doesn’t explicitly address the question at hand—the question of whether American scores are slipping or deteriorating.

That isn’t Ravitch’s fault. In the new book Rich is quoting, Ravitch explicitly says this, about the very TIMSS scores Rich is discussing:

RAVITCH (page 67): So, contrary to the loud complaints from the reform chorus, American students are doing quite well in comparison to those of other advanced nations. Are the scores of American students falling? No. Between 1995 and 2011, the mathematics scores of our students in fourth and eighth grades increased significantly. In science, the scores did not fall; they were about the same. In reading, the scores increased from 2001 to 2011.

Personally, we wouldn’t say that “American students are doing quite well in comparison to those of other advanced nations.” In our view, that statement plainly gilds the lily concerning the Asian tigers.

But that part of Ravitch’s statement doesn’t directly address the claim of “deterioration” or slippage. The next part of her statement does—and Rich didn’t use it.

From there, Rich proceeds to quote Paul Peterson, who makes some gloomy remarks. This provides a hackneyed type of pseudo-journalistic balance, but he too fails to address the specific question—are American scores “deteriorating?” Simply put, Rich doesn’t seem up to the task of keeping her work on target.

This news report was incompetent from its headlines down. Can we talk?

When it comes to reporting on public schools, Rich is functionally illiterate. On its face, this fact seems extremely odd.

The New York Times is our “smartest” newspaper. And Rich, its education reporter, graduated summa cum laude from Yale in the 1990s.

How does work so weak and so lame come from such exalted sources? Answer:

Our nation’s intellectual culture is plainly dead or dying. American students have been doing better. American adult elites are caught in a walking death.

Postscript on Denmark/Germany/France: In the last chunk offered above, Rich presents a textbook example of “on the one hand/on the other hand” pseudo-journalism. She twins an upbeat claim by Ravitch with a gloomy claim by Peterson, failing to see that neither claim speaks directly to the issue at hand.

Beyond that, the statement by Peterson makes it sound like American kids would have been in trouble if France, Denmark and Germany had taken part in that Grade 8 test.

Really? At the Grade 4 level, Denmark and Germany did take part. These are the relevant scores:

2011 TIMSS, Grade 4 math, selected scores:
[...]
Finland 545
England 542
Russian Federation 542
United States 541
Netherlands 540
Denmark 537
Lithuania 534
Portugal 532
Germany 528
[...]
International average 500

With a standard deviation of 100, those scores are fairly tightly bunched. There is little to choose between miraculous Finland, the United States and Denmark.

That said, American students did outscore the Germans and the Danes. The French did not take part.

Why did Rich print that statement by Peterson? This sort of thing is very common in our post-journalistic age.

A cynic would say this: Rich is trying to keep her report in line with a strongly-preferred elite script.

A kinder person would simply suggest that she may be a functional illiterate. That news report was a howling mess from its headlines down.