STALKING THE FAUX EDUCATION REPORTERS: Editing Ryan's report!

WEDNESDAY, NOVEMBER 13, 2013

Part 3—The actual size of the gains: How do our finest young college graduates end up with major journalistic assignments for which they are completely unprepared?

Let’s ask that question a different way: How do unqualified children of our elites end up misreporting the achievements and interests of our low-income children?

Despite the best of intentions, the Atlantic’s Julia Ryan (Harvard ’13) is one such unqualified person. In fairness, she didn’t hire herself.

Ryan is hardly the first of our grossly unqualified “education reporters,” nor will she be the last. That said, how does such an unskilled, unknowledgeable person end up reporting on low-income children for such a famous publication? Why do such people accept such assignments?

A clue to that second question was provided by Ryan herself. Last fall, she wrote a piece for the Harvard Crimson concerning what she had done on her summer vacation.

What did Ryan do on her summer 2012 vacation? As it turned out, she stayed home and slept a great deal!

Good lord! Ryan had come down with mononucleosis, the world-famous “kissing disease.” As she described her long, lazy summer in the Crimson, she took us inside the minds of those who do their prepping at Exeter, then accept important jobs for which they’re plainly unqualified:

RYAN (9/4/12): At some point during these weeks of inactivity, I read Tim Kreider’s “The Busy Trap” in the New York Times, which looked at our culture’s preoccupation with maintaining overscheduled lives. Kreider wrote about adult circles, in which mothers run from boardroom meeting to soccer game and young professionals struggle to squeeze in a drink with friends. But I realized the same thing was happening in my dorm room at college, the place that is supposed to be a responsibility-free break between our high school helicopter parents and our demanding bosses at future jobs.

Harvard students love talking about how busy they are. And many of us really are living life at a breakneck speed, sprinting through classes, extra-curriculars, meals, friendships, reading assignments, and phone calls home. But more than just reflecting a zest for life, being busy is a status symbol among the ambitious. Our hectic schedules and endless commitments mean we must be important. Busyness implies that we have more than our peers—more meetings, more friends, more extra-curriculars, and more drive. It means we are somehow getting ahead of everyone else in a game no one can quite name.

So responding “Nothing,” when asked what I was up to [in the summer of 2012], was humiliating. Even though it was medically mandated, I was ashamed of the absence of an important internship in my life and guilty about my lack of activity. I could hear my résumé calling out from within the depths of my computer, “What about me?”

In her essay, Ryan described the mind-set of the young and the ambitious as they look ahead to “demanding bosses at future jobs.”

“What about me?” So Ryan’s résumé cried as she recovered from her affliction. She was “ashamed of the absence of an important internship” in her life.

What about her? One year later, we have our answer. We find her writing about low-income kids for a major American publication, even though she doesn’t seem to have the slightest idea what she’s talking about.

Her résumé may have called out, saying she had a right to do this. That may help explain the way the country has been misinformed by so many people before her—for example, by Gail Collins and Nicholas Kristof, a pair of very famous faux education pundits.

Good God! In just the last eighteen months, Kristof and Collins have joined forces to give the impression to New York Times readers that wonderful things are occurring in Oklahoma’s schools, while the public schools of Texas present a troubling peril.

In fact, Texas is one of our highest-scoring states on the NAEP, the testing program Collins endorsed in her clueless book about the state last year. And uh-oh! Despite the impression given by Kristof in last Sunday's Times, Oklahoma is one of our lowest-scoring states.

Neither Collins nor Kristof seems to have done any background work before they unloosed their analyses. This is the way our post-journalistic “press corps” typically works.

Given these surroundings, it’s hard to fault an ambitious young person for thinking she has the right to play around with competence questions too. For reporting on an important topic she seems to know nothing about.

Based on Ryan's piece in the Crimson, the ambitious young folk at our finest schools feel the need for high-powered jobs. This doesn’t explain why a publication like the Atlantic would offer Ryan a job for which she’s unprepared.

It does bring us back to her latest bomb, in which she attempted to report on the nation’s 2013 NAEP scores. As she started, Ryan tried to report the size of the nation’s score gains. Let’s explain where this unqualified young person failed:

As we noted in yesterday's post, Ryan raised an obvious question in her hapless report. Have test scores improved a lot or a little over the past twenty years?

She utterly failed to answer that question, for reasons we sketched yesterday. But just for the record, these were the score gains Ryan reported:

Ryan: NAEP score gains since 1990
Grade 4 reading: 5 points
Grade 4 math: 28 points
Grade 8 reading: 8 points
Grade 8 math: 22 points

Are those large score gains? Or are those gains small? Providing no basis for her judgment, Ryan seemed to say the gains in math were large, but the gains in reading were not.

Here’s the rest of the story:

As noted yesterday, Ryan was presenting data for public and private schools combined. We doubt that she realized that. Nor were readers told.

From this point on, we’ll present the data for public schools only. For most NAEP data, start here. For some data, you may have to click this, then click MAIN NDE.

More significantly, Ryan failed to “disaggregate” scores. The demographics of our student population have changed a great deal in the past two decades. This can affect those aggregate scores in certain peculiar ways.

Ryan described a gain of 5 points in Grade 4 reading since 1992. Her statement is technically accurate.

That said, this is what the gains look like if 1) you restrict yourself to public schools and 2) if you look at the score gains achieved by separate parts of the student population. Below, we’ll give you a rule of thumb by which you can start to judge the size of these score gains:

Score gains by public school students, 1992-2013
Grade 4 reading, NAEP
All students: 5.84 points
White students: 8.15 points
Black students: 14.10 points
Hispanic students: 12.06 points

According to Ryan, the average score in Grade 4 reading has risen by 5 points since 1992. That statement can be defended as technically accurate, but here’s what that statement conceals:

In our public schools, the average score by black students in Grade 4 math has risen by 14 points during that period! And oh yes: By a very rough rule of thumb, ten points on the NAEP scale is often compared to one academic year.

Applying that very rough rule of thumb, black kids have made a rather substantial gain in Grade 4 reading during that period.

Alas! Atlantic readers were given no way to assess the 5-point gain which Ryan reported. As usual, they weren't even told about the much larger gain recorded by our black kids.

Ryan went on to lament the "wide achievement gap." She never mentioned the large apparent gains achieved by black and Hispanic kids.

At this point, some readers may think they’ve spotted a problem with the data we’ve presented. If each major group of students gained more than 8 points, how could the overall score gain be less than 6 points?

A competent education reporter should be able to explain that apparent paradox—a paradox which is only apparent. (In all instances, those are the actual score gains.)

We’ll explain that apparent paradox in our next post. For today, let’s review more scores:

Ryan seemed to suggest that the score gains in reading were small. That’s always a subjective judgment, of course. We’d say the apparent gain by black kids is rather large and very much worth reporting.

As we told you last week, reporters always ignore the gains. And good God! Here are the fuller data for Grade 4 math:

Score gains by public school students, 1990-2013
Grade 4 math, NAEP
All students: 29.57 points
White students: 31.29 points
Black students: 37.04 points
Hispanic students: 31.15 points

Applying that very rough rule of thumb, black kids seem to have made enormous gains in math over that period. And by the way: Due to a one-time change in NAEP procedures, all these score gains, including those in reading, are probably understated by a point or two, possibly more.

Starting in 1998, the NAEP began “permitting accommodations” in its testing procedures. This meant that fewer children were excluded from testing due to some sort of language shortfall or physical handicap.

As anyone can see by consulting the data for the transition year, the inclusion of these extra students tended to reduce average scores. Because of that one-time change in procedures, there is no way to make a pure comparison between scores over the past twenty years.

Have black kids gained a year and half in reading at Grade 4? Have they made even larger gains in math? In a world where journalism was occurring, reporters from publications like the Atlantic would pose such questions to NAEP officials.

They would also report these apparent large gains to their journal’s readers.

That would be a journalistic world. We live in a different world, a post-journalistic realm.

In our world, upper-class publications like the Atlantic don't waste their time with questions like these. They tend to report the “wide achievement gap,” as Ryan did. They simply ignore the large apparent progress recorded by minority kids.

Ryan did what everyone does. She reported the gap, understated and ignored the progress. She offered no basis for understanding the size of the score gains at all.

This sort of work is quite unfortunate, but it’s hardly unique to the Atlantic. When Collins and Kristof start limning these topics, they seem to have no earthly idea what they’re doing either. This is very much the way the modern “press corps” works.

Below, you see the fuller data for Grade 8 reading and math. By normal reckoning, the score gains are encouraging in reading, extremely large in math.

What would NAEP officials say about the assessment we’ve just advanced? We don’t know and almost surely we will never find out! The New York Times and the Atlantic display little interest in the progress recorded by black and Hispanic kids. They seem to care more about the ambitious young pseudo-journalists from Nicholas Kristof’s Fair Harvard:

Score gains by public school students, 1992-2013
Grade 8 reading, NAEP
All students: 8.14 points
White students: 9.65 points
Black students: 13.62 points
Hispanic students: 16.28 points

Score gains by public school students, 1990-2013
Grade 8 math, NAEP
All students: 21.88 points
White students: 24.12 points
Black students: 26.61 points
Hispanic students: 26.36 points

Have black kids gained a year and a half in reading, anywhere from two to four years in math? You’d think the public would want to know.

Will anyone ever tell the public about those large score gains by black kids? Will anyone ever start a discussion about what those score gains mean?

It won’t likely happen at the Atlantic. At the Atlantic, nobody seems to care.

Tomorrow: Amanda Ripley on “tracking”