NEWT VERSUS NAEP: The apparent progress of really poor children!

THURSDAY, DECEMBER 15, 2011

PART 4—ONE FACT IS BLATANTLY CLEAR: Last week, the liberal world railed at Newt Gingrich, in part because he said some things a million people had said before him. See THE DAILY HOWLER, 12/14/11.

Gingrich’s rather familiar remarks concerned “really poor children in really poor neighborhoods.” Thundering, roaring and posturing grandly, the mainstream and pseudo-liberal worlds pretended to be extremely disturbed by his Dickensian comments.

But uh-oh! That very same week, the National Assessment of Educational Progress released new data concerning urban school systems. These data concern the life prospects of really poor children in really poor neighborhoods.

The NAEP released a ream of new data—and the “liberal” world didn’t care! Over at Slate, Matt Yglesias did publish a short report about the data, although in our view he did a poor job teasing out the story they told (see THE DAILY HOWLER, 12/12/11). In fairness to Yglesias, he has no mentors or role models in his community who might help him know how to handle such data.

His spirit was willing, but his class background was weak. For the most part, the new data went unexplored, even in the Yglesias piece.

Today, let’s take a look at those new NAEP data, examining the story they seem to tell about the lives of really poor children. Once again, let’s start with Yglesias’ short report about those new test scores:

YGLESIAS (12/7/11): Today the latest round of Trial Urban District Assessment data about American school performance came out. On the reading front it showed no statistically significant change over the past two years in forth [sic] graders in any of the participating cities, and statistically significant (and positive) change for eighth graders in Charlotte while everyone else was insignificant. When you combine all the cities into a large sample, it shows a modest overall statistically significant improvement. In math we [sic] fourth graders improving in Atlanta, Austin, Baltimore, and Philadelphia and eighth graders improving in Atlanta, Charlotte, Chicago, Detroit, DC, and Louisville. There are no statistically significant declines anywhere.

I don't think you can draw any sweeping conclusions from this, but I do think it's worth laying it out there as a baseline. Most of the commentary I read about public education in America is very negative but to the best we can tell things are gradually improving even during a very difficult social and economic period for the country. The math gains in Atlanta seem especially noteworthy since a lot of attention has been paid to a big cheating scandal on the Georgia state tests. It looks, however, like that cheating was occurring against a background of real learning gains for the city's kids.

The thing I do worry about, however, is that lots of cities don't participate in TUDA!. [sic] Policy varies greatly across the participating cities, but one thing they have in common is that they all want to know how they're doing, which I think is the baseline beginning for improvement. But some very large school districts—most notably Clark County in Nevada, a bunch of non-Miami Florida districts, and Dallas, TX—seem to prefer to wallow in ignorance. There's really no excuse for this.

That was Yglesias’ full report, complete with at least three typos. When we liberals write about really poor children, we don’t seem to proof-read our work with great care. Our editors may not bother to read such tedious postings at all.

First, let’s look at the basic points Yglesias made. Then, let’s look at the data to which his piece links.

For starters, let’s consider what he said in his third, and final, paragraph.

Yglesias made an accurate if somewhat minor point in that closing paragraph, in which he discussed "the thing I do worry about." It's true: The NAEP’s so-called Trial Urban District Assessment does not present data from all the nation’s big urban school systems. By our count, only nine of the nation’s twenty largest cities are currently taking part in the project. That said, we have no idea why Yglesias singled out the non-participating cities he did. Dallas is a major non-participant, but Phoenix and San Antonio are larger by population; among non-participating Florida cities, only Jacksonville and Tampa seem to qualify for the project on the basis of size. Meanwhile, a long list of other big cities don’t take part in the study, including Las Vegas, which is part of Clark County, one of the nation’s largest school districts. Ranked by population, from the largest on down, these are the biggest cities which don’t take part in the project:

Phoenix, San Antonio, Dallas, San Jose, Jacksonville, Indianapolis, San Francisco, Columbus, Fort Worth, El Paso, Memphis, Seattle, Nashville, Denver, Portland (Oregon), Las Vegas, Oklahoma City. Also, among our “major-league” cities: Oakland, Kansas City, St. Louis, Pittsburgh, Oakland, Minneapolis, Cincinnati, New Orleans.

We don’t know the basis on which Yglesias selected the cities he cited, although there may be some such basis. But so it goes when liberal writers, deprived of mentors and working role models, write about topics like this.

That was a fairly minor point, although it ate a third of Yglesias’ report. His summary of the data to which he linked was vastly more significant. Yglesias focused on test score changes from 2009 to 2011, for forth (sic) and eighth grade students alike. Beyond that, he briefly considered the change which occurred “when you combine all the cities into a large sample,” although we think he is probably mis-describing the data he’s reviewing in that case. His basic judgment: “I don't think you can draw any sweeping conclusions from this, but I do think it's worth laying it out there as a baseline.” And this: “To the best we can tell things are gradually improving even during a very difficult social and economic period for the country.”

We agree with that first point. It’s hard to draw sweeping conclusions from data like these; it’s dangerous to draw such conclusions if you care about being right. But it’s especially hard to draw sweeping conclusions when we limit ourselves to a two-year span and focus on individual districts. These are the reading data to which Yglesias linked in his report; these are the corresponding math data. If you simply perform one or two more clicks in each instance, you come to some truly useful data tools. These data tools let us draw larger semi-conclusions about the state of progress being displayed by really poor children.

(For example: One more click from that math page takes us to this remarkably useful data tool, a tool which is swimming with data.)

How are really poor children in really poor neighborhoods doing on the NAEP? Ignore those district-by-district scores, which deal with relatively small, discrete populations. Instead, let’s focus on the nationwide data for various groups. On a national basis, how are black students doing? How about Hispanic kids? And wow! Those new data tools break low-income students down into two separate groups! We may be wrong, but we don’t think we’ve ever seen NAEP data do that before.

Let’s select a significant time span. Let’s see how various groups are doing, on a nationwide basis, in the eight years from 2003 through 2011. As we do, let’s recall that very rough rule of thumb: Ten points on the NAEP scale is often said to be roughly equal to one academic year:

Fourth grade math: According to those data, black fourth-graders gained eight points in math during that eight-year span. So did Hispanic fourth-graders. Kids receiving reduced-price lunch gained nine points during that span. (This includes children of all races and ethnicities.) Kids receiving free lunch (the “really poor children”) gained eight points too.

Fourth grade reading: Black fourth-graders gained eight points. Hispanic students gained six points. Reduced-price students gained seven points. Free lunch kids also gained seven.

Eighth grade math: Black eighth-graders gained ten points. Hispanic students gained eleven. Reduced-price kids gained ten points. Free lunch kids gained twelve.

Eighth grade reading: Black eighth graders gained four points. Hispanic students gained seven points. Reduced-price students gained five point. Free lunch kids gained seven.

To the extent that we trust that very rough rule of thumb, those are substantial score gains; some of the gains are just large. And again: Those are the gains displayed by national samples of these groups, based on the NAEP’s nationwide testing.

Please note: Free lunch kids, our “really poor children,” exhibited score gains too.

Children who receive free lunch are the poorest children on whom we have data. As you can see if you look at those data, they score lower in reading and math than the reduced-price students do; they score well below the kids who don’t get a price break at all. “Really poor children in really poor neighborhoods” don’t score as well as their more advantaged peers. But they seem to be making progress, although we’d like to see one of the nation’s handful of journalists interview major NAEP officials about those data—about the conclusions we might want to draw from those data, if we want to be careful and right.

Really poor children in really poor neighborhoods seem to be making progress. That said, one other fact is just blatantly clear:

The liberal world doesn’t care.

Tomorrow: Professor Gates said it, not us!