PEDDLING RHEE: Fuzzy claims, no actual numbers!

TUESDAY, JANUARY 15, 2013

Part 2—What happened to Washington test scores: Did test scores rise in the DC schools because of Michelle Rhee's “reforms?”

Last weekend, readers of the Washington Post were told that test scores did go up. But they were offered no data.

(For part 1 in this award-winning series, see yesterday's DAILY HOWLER.)

Over the weekend, the Post went into Full Rhee Mode, as is its periodic wont. In a trio of reports, the paper defined the wonders of the nation’s new “education celebrity” while defending her against claims of cheating on the DC-CAS, the District’s own annual testing program.

Rhee was everywhere in the Post! The former head of the DC schools was the subject of a major profile at the top of Sunday’s front page. Beyond that, she was defended against accusations of test score tampering in a Saturday editorial, and in a Sunday opinion piece by her biographer, Richard Whitmire.

In its editorial, the Post made a rather nuanced defense of the accusations about cheating. The editors said that a newly completed federal probe “found no evidence that D.C. school officials engaged in widespread cheating.”

In our view, the editorial and the Whitmire piece glossed the way Rhee’s aggressive focus on test scores may have encouraged cheating by teachers and principals, in ways she certainly should have foreseen. But let’s forget about that for now! As the editors proceeded, they offered the following claim:

WASHINGTON POST EDITORIAL (1/12/13): Questions about [possible cheating on] the [District’s own] test don't change the larger story about Ms. Rhee's record. D.C. students made significant progress during her tenure as measured by the National Assessment of Educational Progress (NAEP), a test on which no one has alleged cheating.

According to the editors, DC students “made significant progress” on the NAEP, the widely-praised “gold standard” of educational testing.

“Significant” progress sounds like a good thing! That said, the editors presented no actual data in support of their upbeat assertion. In his own opinion piece about Rhee, Whitmire expanded the claim about test score gains while describing the NAEP in more detail:

WHITMIRE (1/13/13): Most of the media coverage over cheating ignores something fundamental: The controversy is over cheating on the D.C. CAS (Comprehensive Assessment System), the local exam whose results are used to reward or punish teachers and principals.

This test has nothing to do with the federal exam used to compare achievement by D.C. students to similar urban students around the country. That test, the National Assessment of Educational Progress (NAEP), is the so-called "gold standard" of testing, and it showed that D.C. students made unique progress during the Rhee years.

It is almost impossible to cheat on the NAEP, which is administered by the federal government. In fact, there has never been any evidence of cheating on that test. Besides, there is no motive to cheat on the NAEP: No jobs are at stake.

Whitmire is certainly wrong on one point. By now, superintendents—people like Rhee—do have an incentive to cheat on the NAEP, although we’ve never seen any suggestion that anyone has ever done so, nor do we know if they could. (Based on past behavior, the press corps will start to examine this question at some point around the year 2080.)

At any rate, Whitmire amplified the editors’ claim about score gains on the NAEP—though his claim that students made “unique” progress is bit hard to interpret. But uh-oh!

Like the editors, Whitmire presented no actual test score data in support of his upbeat claim. Later, he returned to this general theme:

WHITMIRE: This rush to diminish Rhee's legacy sidesteps the question of how that legacy should be defined. I decided to write a book about Rhee because I thought her reforms might answer this question: Is it possible to take an urban school district designed almost solely for adults and refocus the system on kids?

At first, Rhee's reforms appeared to be working. Low-income black students really did start to do better. As it turns out, the many D.C. teachers who for years blamed the shortcomings of their students entirely on their impoverished home lives were only partly right. Schools, at least those that know how to promote effective teaching, can make a difference.

Presumably, Whitmire refers to the NAEP when he says that “low-income black students really did start to do better” in response to Rhee’s policy changes.

Let’s stop to review:

According to the editors, DC students made “significant progress” on the NAEP because of Rhee’s interventions. Whitmire used a stranger term—the progress was “unique.”

But how odd! Neither of these presentations included any actual test score data! Readers were told that test scores rose, in “significant” ways. But there was no attempt to show them the money—to use any actual numbers.

Meanwhile, on Sunday’s front page, Lyndsey Layton had 2100 words to describe Rhee’s rise to celebrity status. Layton didn’t assert that test scores rose under Rhee; essentially, she skipped this topic altogether.

She did present the views of Richard Kahlenberg, a critic of Rhee. But doggone it! In Layton’s presentation, this is what Kahlenberg said:

LAYTON (1/13/13): "She's got a very simple message that is highly seductive because it appears to give an answer to our difficult education problems," said Richard Kahlenberg, a senior fellow at the Century Foundation, a liberal-leaning research group.

It would be great if her ideas translated into good results for kids, Kahlenberg said.

"But, in fact, we've got two grand experiments of her theory," he said. "The first is the American South, where teachers unions are weak and the schools are not lighting the world on fire. The other is charter schools, which are 88 percent non-unionized. In charters, you can do everything that Michelle Rhee wants to do—fire bad teachers, pay good teachers more. And yet, the most comprehensive studies looking at charter schools nationally find mediocre results."

So Rhee's premise is faulty, he said. "But it's a simple idea, and in the media, it's powerful to have heroes and villains," Kahlenberg said. "The fact that evidence doesn't back her up doesn't seem to prevent her from getting wide notoriety."

Kahlenberg discussed “two grand experiments” of Rhee’s ideas, one of which is fairly silly. But he didn’t discuss the simplest experiment of all:

At least as quoted by Layton, Kahlenberg didn’t mention the actual test scores which actually came out of DC schools. Three different reports discussed Rhee’s tenure—and none of them showed you the money!

What actually happened in DC schools during and after Rhee’s tenure? More specifically, what happened to scores on the NAEP, the testing program explicitly cited by Whitmire and by the editors?

How odd! The Post fell all over itself this weekend, praising Rhee and asserting that students showed “significant progress” on the NAEP. But no one made any attempt to present any actual numbers!

Increasingly, so it goes when Rhee is discussed, and in the American press corps in general. Increasingly, our public discourse is defined by the things the public isn’t told.

In this case, an obvious dog neither barked nor yelped. What actually happened to DC’s scores?

Tomorrow, a look at the record.

Tomorrow: The aforementioned look at the record!