We return to that New York Times news report!

SATURDAY, NOVEMBER 2, 2013

Motoko Rich, functionally illiterate: You’re right!

This past week, we never got back to that news report in the New York Times about international test scores.

The report appeared on October 24, atop the front page of the National section. It was written by education reporter Motoko Rich.

For our first post on this subject, click here.

Our question was this: How is it possible that someone like Rich could write such an utterly clueless report? Given our focus on Minnesota’s test scores, we never got back to that topic in our posts this week.

If you want to puzzle about your nation’s journalistic culture, we’ll suggest that you peruse that report. Focus on three topics:

Clueless from the headline down: Rich’s report concerned a new study of U.S. math and science scores on the 2011 TIMSS. We marveled at the headlines on the hard-copy report, and at its opening paragraph:
RICH (10/24/13): Better News In New Study That Assesses U.S. Students/
Majority Outperform International Average


Amid growing alarm over the slipping international competitiveness of American students, a report comparing math and science test scores of eighth graders in individual states to those in other countries has found that a majority outperformed the international average.
The headlines mirror Rich’s opening paragraph, although they may miss one nuance. That said, those headlines are quite hapless. Here’s why:

From those headlines, a reader would think it came as “news” to learn that a majority of U.S. students outperformed the international average on the tests in question.

In fact, TIMSS results from 2011 were released last December. Since that time, it has been known that American students outperformed the international average in math and science at both grade levels tested.

On the TIMSS scale, the international average is set at 500. These were the United States’ average scores:
Average scores, United States, 2011 TIMSS
Grade 4 math: 545
Grade 8 math: 509

Grade 4 science: 544
Grade 8 science: 525
Grade 8 math was the weakest area for American students. But everyone has known since last December that American students outscored the international average on all four TIMSS tests.

Everyone has known that except the New York Times’ readers! At the Times, Rich wrote a confusing report last December about the new TIMSS scores. In this latest report, she and the Times seemed to think it was “news” that U.S. students had exceeded the international average.

(In her opening paragraph, Rich may mean that “a majority” of states “outperformed the international average.” Given the average scores for the nation as a whole, it's hard to know why even that would be surprising.)

It’s hard to get more clueless than this. But Rich and her unnamed editors were equal to the task.

Reinforcing a ubiquitous bogus script: Let’s return to that opening paragraph. In her very first sentence, Rich reinforces a bogus, ubiquitous script:
RICH: Amid growing alarm over the slipping international competitiveness of American students, a report comparing math and science test scores of eighth graders in individual states to those in other countries has found that a majority outperformed the international average.
Has the international competitiveness of American students been “slipping?” That is, of course, a ubiquitous claim. But is this famous claim true?

As far as we know, there is no international forum in which scores by American students have been “slipping.” (Or “deteriorating,” the harsher term Rich employs a bit later on.)

American test scores could surely be better. And many folk do express alarm about the idea that those scores are “slipping.”

But in fact, American scores have not been “slipping” or “deteriorating.” Unless you read the New York Times, a famous newspaper which seems determined to keep confusion alive.

Can’t stick to the question at hand: As she continues, Rich described the findings of the study on which she was reporting. Near the end of her report, she returned to the claim that U.S. scores have been “deteriorating” in recent years.

In this passage, Rich seems to examine that claim. She seems to examine that claim but, in various ways, she fails. The journalistic work in this passage is just astoundingly bad:
RICH: A growing chorus of critics have said that claims of deteriorating American student performance have been exaggerated. In her recent best seller, “Reign of Error,” the education historian Diane Ravitch wrote, “Contrary to the loud complaints from the reform chorus, American students are doing quite well in comparison to those of other advanced nations.”

But others say the international comparisons are apt. Paul E. Peterson, the director of the program on education policy and governance at Harvard and a fellow at the conservative-leaning Hoover Institution, said the study provided less good news than people might think.

He pointed out that the pool of countries taking the math and science exams included many developing countries, and that several industrialized nations, including France, Germany and Denmark, did not participate. “So if you really want to compare the U.S. to the developing world, then we do look good,” Mr. Peterson said.
Truly, that is awful. Start with the first paragraph:

“A growing chorus of critics have said that claims of deteriorating American student performance have been exaggerated,” Rich writes. To many, this presentation will convey the idea that student performance has been deteriorating. It’s just that the degree of deterioration has been exaggerated.

In fact, there has been no “deterioration” at all. A competent journalist would simply say that.

As she continues, Rich quotes a statement by Diane Ravitch. This statement doesn’t explicitly address the question at hand—the question of whether American scores are slipping or deteriorating.

That isn’t Ravitch’s fault. In the new book Rich is quoting, Ravitch explicitly says this, about the very TIMSS scores Rich is discussing:
RAVITCH (page 67): So, contrary to the loud complaints from the reform chorus, American students are doing quite well in comparison to those of other advanced nations. Are the scores of American students falling? No. Between 1995 and 2011, the mathematics scores of our students in fourth and eighth grades increased significantly. In science, the scores did not fall; they were about the same. In reading, the scores increased from 2001 to 2011.
Personally, we wouldn’t say that “American students are doing quite well in comparison to those of other advanced nations.” In our view, that statement plainly gilds the lily concerning the Asian tigers.

But that part of Ravitch’s statement doesn’t directly address the claim of “deterioration” or slippage. The next part of her statement does—and Rich didn’t use it.

From there, Rich proceeds to quote Paul Peterson, who makes some gloomy remarks. This provides a hackneyed type of pseudo-journalistic balance, but he too fails to address the specific question—are American scores “deteriorating?” Simply put, Rich doesn’t seem up to the task of keeping her work on target.

This news report was incompetent from its headlines down. Can we talk?

When it comes to reporting on public schools, Rich is functionally illiterate. On its face, this fact seems extremely odd.

The New York Times is our “smartest” newspaper. And Rich, its education reporter, graduated summa cum laude from Yale in the 1990s.

How does work so weak and so lame come from such exalted sources? Answer:

Our nation’s intellectual culture is plainly dead or dying. American students have been doing better. American adult elites are caught in a walking death.

Postscript on Denmark/Germany/France: In the last chunk offered above, Rich presents a textbook example of “on the one hand/on the other hand” pseudo-journalism. She twins an upbeat claim by Ravitch with a gloomy claim by Peterson, failing to see that neither claim speaks directly to the issue at hand.

Beyond that, the statement by Peterson makes it sound like American kids would have been in trouble if France, Denmark and Germany had taken part in that Grade 8 test.

Really? At the Grade 4 level, Denmark and Germany did take part. These are the relevant scores:
2011 TIMSS, Grade 4 math, selected scores:
[...]
Finland 545
England 542
Russian Federation 542
United States 541
Netherlands 540
Denmark 537
Lithuania 534
Portugal 532
Germany 528
[...]
International average 500
With a standard deviation of 100, those scores are fairly tightly bunched. There is little to choose between miraculous Finland, the United States and Denmark.

That said, American students did outscore the Germans and the Danes. The French did not take part.

Why did Rich print that statement by Peterson? This sort of thing is very common in our post-journalistic age.

A cynic would say this: Rich is trying to keep her report in line with a strongly-preferred elite script.

A kinder person would simply suggest that she may be a functional illiterate. That news report was a howling mess from its headlines down.

12 comments:

  1. "A cynic would say this: Rich is trying to keep her report in line with a strongly-preferred elite script."

    I'd be willing to be she's just lazy.

    It makes me wonder: has anyone on the education beat at the NYT ever known how to analyze test scores? When someone is moved to the education beat, as we know Rich was not long ago, are they given any training or expected to just wing it?

    ReplyDelete
    Replies
    1. Sorry, meant to type "I'd be willing to bet she's just lazy."

      Delete
    2. This comment has been removed by the author.

      Delete
    3. I made the exact same typo as Anonymous. What I was meant to type was:

      I'd be willing to bet that when a reporter is assigned to write an education article, the editor gives the reporter a list of people to call for expertise and comments. The reporter, who knows little or nothing about test scores, gets input from those people and turns it into a paragraph or two.

      Delete
  2. Where's that shit KZ? I don't know what to think 'til I read his profound, clever droppings. I mean, I know Somerby's wrong, of course. And bad. But I just can't *think* it without KZ's helpful heap.

    ReplyDelete
    Replies
    1. Gee, I dunno. Maybe he's got a life outside this blog?

      Delete
    2. Anonymous November 2, 2013 at 6:23 PM:

      Leave now.

      Delete
  3. I still find it very, very odd that even Somerby does not compare the US with Russia, which seems to do better on the TIMSS. Have all their school systems been converted to privatized voucherfests? What is going on there? How do they manage to keep up despite the breakup of the socialist paradise as well as that of the former Russian Empire? Russia is still a huge country, which would seem to be a better country to compare with than tiny Singapore or even Finland.

    ReplyDelete
  4. I seem to have some ancestral memories of how after the launching of Sputnik the US educational system was compared to that of the USSR. How have things changed since then?

    ReplyDelete
  5. "In fact, there has been no deterioration at all. A competent journalist would simply say that."

    No, even that's just one more way to be unaccountable. A competent journalist would report, as news, that conventional wisdom has been wrong for years, then comment on that fact extensively. The fact that it's not new doesn't mean it's not news if there are decades of cowardly stupidity surrounding a topic.

    It's like how one day Bush was a great leader, and the next, "everyone knows" he's a giant screwup so let's not belabor that. At no point did the media sit down, day after day for months, and report as news the fact that their judgment was awful.

    ReplyDelete
  6. I'd say she may be incompetent at telling the truth, but she's a competent liar. It wouldn't be the first time the "paper of record" has published lies. -E

    ReplyDelete
  7. "Our nation’s intellectual culture is plainly dead or dying. American students have been doing better. American adult elites are caught in a walking death."

    Thank God those American adults living in a dying or plainly dead intellectual culture have been doing such a good job improving our youth.

    ReplyDelete