PEDDLING RHEE: What the NAEP scores actually say!

WEDNESDAY, JANUARY 16, 2013

Part 3—The Post’s idea of progress: Last weekend, the Washington Post was engaged in its favorite task.

The Post was peddling Rhee.

On Sunday, a 2100-word front-page profile introduced the world, for the ten millionth time, to its new “education celebrity.” Meanwhile, an editorial and a separate opinion piece told the world, for the ten millionth time, about the progress Michelle Rhee achieved in just three years as head of DC’s public schools.

(For part 2 in this award-winning series, see THE DAILY HOWLER, 1/15/13.)

As usual, the Post was peddling its top educational savior. The paper downplayed the obvious cheating which occurred on district-wide, high-stakes tests during Rhee’s rocky tenure.

Instead, the Post focused on the National Assessment of Educational Progress (the NAEP)—and it made pleasing claims in its editorial about the progress displayed on those tests. In his separate opinion piece, Richard Whitmire did likewise:

WASHINGTON POST EDITORIAL (1/12/13): Questions about [possible cheating on] the [District’s own] test don't change the larger story about Ms. Rhee's record. D.C. students made significant progress during her tenure as measured by the National Assessment of Educational Progress (NAEP), a test on which no one has alleged cheating.

WHITMIRE (1/13/13): Most of the media coverage over cheating ignores something fundamental: The controversy is over cheating on the D.C. CAS (Comprehensive Assessment System), the local exam whose results are used to reward or punish teachers and principals.

This test has nothing to do with the federal exam used to compare achievement by D.C. students to similar urban students around the country. That test, the National Assessment of Educational Progress (NAEP), is the so-called "gold standard" of testing, and it showed that D.C. students made unique progress during the Rhee years.

It is almost impossible to cheat on the NAEP, which is administered by the federal government. In fact, there has never been any evidence of cheating on that test. Besides, there is no motive to cheat on the NAEP: No jobs are at stake.

“At first, Rhee's reforms appeared to be working,” Whitmire wrote a bit later. “Low-income black students really did start to do better.”

As usual, the Post was peddling Rhee to the world. But how odd! The Post and Whitmire each described the “significant,” “unique” progress DC students displayed on the NAEP during Rhee’s reign.

But neither piece cited any actual test scores—actual numbers which would let a reader judge how “significant” that progress had been.

How large were the score gains under Rhee? Readers kept reading the old words of praise. But they were given no numbers.

Today, let’s take a look at the record! As best we can, let’s look at the actual NAEP scores achieved in DC before and after Rhee’s three-year reign.

Did DC students really show “significant progress” as a result of Rhee’s tenure? Did low-income black students “really start to do better?”

In our opinion, the Post significantly distorted the record with its familiar upbeat claims. But let’s take a look at the actual record. Let’s take a look at those test scores!

First, a note on method: The NAEP, a widely-praised federal program, is administered every other year. Most recently, U.S. students were tested in reading and math in 2007, 2009 and 2011.

Rhee assumed her post in DC in June 2007; she resigned in October 2010. She was succeeded by Kaya Henderson, her top aide, who continued her policies.

As such, the 2007 NAEP testing immediately predated Rhee’s tenure in Washington. The 2011 testing followed her three years at the helm, along with part of Henderson’s first year. If we compare the 2007 and 2011 scores, we get an imperfect but pretty good look at the progress which may have occurred during this time.

Our reaction to these test scores? If this is the Post’s idea of “significant progress,” the Post’s ideas are not good.

Let’s start with reading:

In reading as well as in math, DC's black students were scoring well below their peers in other big-city systems when Rhee arrived on the scene. But uh-oh! According to the NAEP test scores, the same situation obtained in reading four years later. Below, you see the average scores for black students in DC’s non-charter schools—in the schools Rhee supervised.

In fourth grade, the average reading score stayed where it was on the NAEP. In eighth grade, the average declined:

Average scores, DC black students in non-charter schools, NAEP reading:
Fourth-graders, 2007: 191
Fourth-graders, 2011: 191

Eighth-graders, 2007: 233
Eighth-graders, 2011: 231

Rather plainly, that isn’t “significant progress”—although judged as progress, this type of movement may qualify as “unique.”

(All these data can be obtained from the NAEP Data Explorer. Just click here, then click on MAIN NDE. After that, you’re on your own.)

In math, there were score gains. On the eighth-grade level, the score gains were quite substantial:

Average scores, DC black students in non-charter schools, NAEP math:
Fourth-graders, 2007: 208
Fourth-graders, 2011: 212

Eighth-graders, 2007: 240
Eighth-graders, 2011: 249

That eighth-grade score gain is quite substantial. Let’s offer the usual rule of thumb along with some words of caution:

By a well-known but very rough rule of thumb, ten points on the NAEP scale is often compared to one academic year. This is a very rough rule of thumb. But it gives us a very rough way of judging the possible meaning of those score gains.

A nine-point score gain in four years is a significant gain. Black eighth-grader around the nation only gained three points during this period, although they remained far ahead of their peers in DC:

Average scores, black students nationwide, NAEP math:
Fourth-graders, 2007: 222
Fourth-graders, 2011: 224

Eighth-graders, 2007: 259
Eighth-graders, 2011: 262

As you can see, black eighth-graders in DC still trailed their nationwide peers by a healthy margin in the 2011 testing. That said, it would be foolish to expect someone like Rhee to enter the DC schools and erase the large achievement gaps which existed before she arrived in just three or four years.

Other factors are at play in the DC scores. For example, the large number of charter schools in DC adds an element of uncertainty. If students switch between charters and non-charters, this may produce changes in average scores which represent a changed demographic, rather than changes in instruction. Meanwhile, DC schools excluded a higher percentage of students from the testing in 2007 and 2009 than they did in 2011 (due to disabilities and language issues). Theoretically, a change of this type could affect average scores too.

But there you see some of the actual scores achieved on the actual NAEP. What sort of “progress” is seen here?

To judge from the NAEP average scores, black students in DC made no progress in reading—none at all—as a result of Rhee’s tenure.

Personally, we wouldn’t “blame” Rhee for that. It isn’t easy to improve reading performance among deserving kids from low-literacy backgrounds, and Rhee has never shown any sign of having any ideas about how to improve instruction.

That said, we would blame the Washington Post for its unending pimping and propaganda. Judging from the “gold standard” NAEP, DC’s black students did seem to improve in math during Rhee’s tenure—although DC’s horrible scores on the NAEP were already rising in the years before Rhee arrived.

But DC’s black students showed no progress in reading during the period in question. The Post should be ashamed of itself for withholding such information even as it pimps its favorite “education celebrity” to a disinformed world.

We like Rhee for some of her attitudes. In other ways, her attitudes are disastrous, bizarre—and she has never shown any sign of having ideas about instruction.

That said, she’s the dream girl of the world’s billionaires—and the Washington Post won’t stop peddling.

Tomorrow: Disgraceful disinterest, neglect