The Washington Post is confused by NAEP scores!

MONDAY, MARCH 3, 2014

In the wake of the Howard Beale Show: We’ve often mentioned the liberal world’s indifference to the interests of low-income children.

To all intents and purposes, the interests of low-income kids never get mentioned on MSNBC. Issues surrounding urban and low-income schools are completely and wholly ignored.

Out in the wider press corps, how thoroughly does no one care? Consider the bungling in a news report in last Friday’s Washington Post.

At the start of Emma Brown’s report, a DC council member was complaining the good complaint—or at least, so it seemed.

He complained about the achievement gaps in the DC schools. He said DC officials were leading cheers for very limited progress:
BROWN (2/28/14): D.C. Council member David A. Catania recited a litany of data Thursday to illustrate the city's large and persistent student achievement gaps, using an annual oversight hearing for the school system to ask whether officials are moving quickly enough to improve outcomes for poor and minority children.

"What I hear is this constant cheerleading…about this fantastic trajectory we're on" without a straightforward accounting of how disadvantaged students are faring, said Catania (I-At Large), who is chairman of the council's Education Committee and is contemplating a run for mayor.
In large part, DC has unusually large “achievement gaps” due to its highly affluent, high-SES white student population. As Brown continued, she made a gloomy claim about progress in recent years—a gloomy claim that is basically wrong:
BROWN (continuing directly): The D.C. school system had the country's largest math and reading gains on the 2013 National Assessment of Educational Progress, growth that drew widespread attention and praise from Education Secretary Arne Duncan.

But Catania said that the overall gains on national tests have largely been driven by population shifts that have left schools whiter and more affluent. Since 2007, for example, reading scores for fourth-grade black students have not budged, and the gap between poor and affluent children has grown.
Is that true? Is it true that reading scores for DC’s black fourth-graders haven’t budged since 2007?

If true, that’s dispiriting news. If true, that would undercut the claim that significant score gains have occurred in DC.

Basically, though, it isn’t true. Presumably, this is why Brown and/or Catania may have thought it was:

Because of the growth of charter schools in DC, it can be tricky to keep track of the District’s test scores. Over the years, a lot of kids have migrated from DC’s traditional public schools into DC’s charter schools.

These transfers aren’t occurring at random. As liberals have noted in other contexts, the kids who choose to transfer to charters may be more ambitious and serious, on average, than the kids they leave behind in traditional public schools.

Let’s consider the reading scores of black fourth graders in the city’s traditional public schools. In fact, there has been a minor gain over the period in question:
Average reading score, Grade 4 NAEP
Black students, DC traditional public schools only

2013: 192.42
2007: 191.24
Within the traditional public schools, the average score was a little more than one point higher in 2013.

No, that isn’t much of a gain. But remember, that gain was achieved even as kids were leaving the traditional schools in favor of the charters.

That is a modest gain. But here’s what the scores look like if you consider all black kids in the D.C. schools—kids in traditional public schools plus kids in the charters:
Average reading score, Grade 4 NAEP
Black students, all DC schools

2013: 196.89
2007: 191.54
That’s a gain of 5.35 points in six years, a pretty good gain. (By a very rough rule of thumb, ten points on the NAEP scale is often compared to one academic year.)

Where did Brown and/or Catania get the idea that “reading scores for fourth-grade black students have not budged” since 2007? We’ll guess it happened like this:

The NAEP does a very poor job presenting DC scores, especially before 2009. If you accept the default presentation the NAEP makes—if you don’t know how to separate scores into charter schools versus traditional public schools—you may see this comparison for “DCPS” in the years in question:
Average reading score, Grade 4 NAEP
Black students, “DCPS”

2013: 192.42
2007: 191.54
In fact, that average score for 2007 is the average score for all kids in all DC schools, traditional public schools plus charters. The score for 2013 is the score for kids in traditional public schools only.

That’s a shaky type of comparison. (In effect, it's apples to apples-plus-oranges.) Meanwhile, if you don’t get the scores to two decimal places, you’ll get this rounded result:
Average reading score, Grade 4 NAEP
Black students, “DCPS”

2013: 192
2007: 192
From that, you may get the impression that “reading scores have not budged.” That is not the case.

A lot of the problem here lies with the NAEP, which presents these data for DC in a confusing way. But there’s no excuse for the Washington Post—for Brown’s failure to understand how to work with these data.

If we look at all black kids in DC schools, fourth grade reading scores have risen at a fairly good pace in the years in question. As of Friday, the Washington Post didn’t know that. It didn’t know how to make apples-to-apples comparisons over that span of time.

In our previous post, we mentioned the press corps’ refusal to note the ways it has turned into a version of the Howard Beale Show. Over the years since Network appeared, all types of obvious clowning have been accepted with barely a word of complaint.

In this matter, we see the flip side of that descent. As cable news has made a joke of the American public discourse, our biggest newspapers don’t even know how to access and assess local test scores.

On liberal cable, they hand us bullshit about New Jersey, a point we’ll continue to note all week. They wouldn’t stoop to discuss black kids if their own lives were at stake.

When our famous Rhodes Scholars behave in such ways, what hope is there for the world?

For all NAEP data: For all NAEP data, click here, then click on MAIN NDE. Click to agree to the usage agreement.

After that, you’re on your own. Warning! In the somewhat unusual case of DC, the NAEP doesn’t make this easy.

Warning: Within the NAEP data, “DCPS” means one thing; “District of Columbia” means something else. Some practices changed in 2009. When it comes to DC schools, the NAEP doesn’t make this easy.

14 comments:

  1. "On liberal cable...They wouldn’t stoop to discuss black kids if their own lives were at stake." BS 3/3/14

    Wow! Things have improved.

    "But the children on The One True Channel would rather jump off the Eiffel Tower than stoop to the level of discussing our ratty public school students." 9/5/2013

    ReplyDelete
  2. How thoroughly does no one care? That’s a peculiar question, one which may lead us down an extremely murky path. According to Bob that peculiar question leads to a post with a succession of endlessly dense sentences, a post which leaves his readers hopelessly confused. The NAEP doesn't make this easy.

    Fuzzy concepts don’t get clearer by injecting adjectives into the stew.

    ReplyDelete
    Replies
    1. They don't care enough to assign an education writer with training in how to interpret educational testing results. How's that?

      Delete
    2. No, 5:27's right: 5:27's an idiot who can't figure this out.

      Delete
    3. Anon @5:27 was imitating Somerby's style, in particular that aspects of Somerby's style that tend to obscure the point.

      Delete
  3. Bob wrote; ...if you consider all black kids in the D.C. schools—kids in traditional public schools plus kids in the charters:

    It's not quite true that: Black kids in traditional public schools + black kids in the charters = all Black kids. There are also black kids in private and parochial schools.

    ReplyDelete
    Replies
    1. It equals all black kids for purposes of the NAEP because the kids in private and parochial schools don't take that test, do they?

      Delete
  4. Eugene Robinson, author of Disintegrated (2010), a book about Black America, calls urban African Americans of the underclass "The Abandoned." It isn't just Somerby's contention that this segment of our school population is being ignored. Demonstrating care means more than saying that you care. Demonstrating care might involve focusing on issues relevant to the US underclass more often on MSNBC. For a reporter it might involve caring enough to disaggregate properly and see whether underclass black children are improving, not just their middle class or aspiring black agemates. It is possible to ignore the underclass better if you lump them in with black children who are not part of the underclass (those from charter schools for example or those from gentrified neighborhoods in DC). Words without active demonstration of caring mean little -- here is where I depart from Obama's famous speech about "just words" to say that words can be empty when unaccompanied by actions.

    ReplyDelete
  5. "We’ve often mentioned the liberal world’s indifference to the interests of low-income children. "

    It is disgraceful, isn't it? If only someone in the liberal world would advocate for policies that would provide food and medical care for low-income children. It's just shameful, the complete lack of regard the liberal world has for low income children.

    ReplyDelete
  6. "the complete lack of regard the liberal world has for low income children." Odd. I would swear the rich had that perception. Excuse me if I do not get excited about a variation which cannot be presumed to have reached the status of significant figures. Positing accuracy on such a 'reading' tends to make one think of self flagellation over a mirage.

    ReplyDelete
  7. Thanks for your interpretation, Bob. I always rely on your comments in this matter.

    LTR

    ReplyDelete
    Replies
    1. Also, Bob, I cannot bear reading comments because of the trolling. Possibly you could ban a few trolls and this would stop the problem immediately. Krugman has done that.

      LTR

      Delete
    2. On no other blog will you see comments as obsequious as you consistently see here. What, anyone who disagrees and gets sarcastic about it is a troll?

      Delete
    3. In the good old days of TDH a comment this late would be spam.

      Delete