EXPERTS, PROFESSORS AND JOURNOS: Fictitions!

TUESDAY, SEPTEMBER 15, 2015

Part 2—Score gains in Grade 12 math: Have high school seniors been doing less well in math in recent years?

In theory, that’s an important question. In practice, such questions mainly exist to let our journalists and “educational experts” repeat standard establishment narratives:

Nothing is working in our schools! Our teachers, with their infernal unions, have foiled education reform!

Have high school seniors been doing less well in math? For various reasons, it’s a hard question to answer. But as you may recall, the Washington Post seemed to think it was up to the task on Thursday, September 3.

That morning, the paper offered a gloomy, front-page report as part of our annual back-to-school pseudo-reporting. With apologies, we’ll post its opening nugget again, as we did last week.

Nick Anderson did the reporting. Needless to say, a leading “educational expert” was instantly cited as Anderson offered gloomy thoughts about the performance of high school seniors, both on the SATs and on some unnamed “federal tests:”

ANDERSON (9/3/15): Scores on the SAT have sunk to the lowest level since the college admission test was overhauled in 2005, adding to worries about student performance in the nation’s high schools.

The average score for the Class of 2015 was 1490 out of a maximum 2400, the College Board reported Thursday. That was down 7 points from the previous class’s mark and was the lowest composite score of the past decade. There were declines of at least 2 points on all three sections of the test—critical reading, math and writing.

The steady decline in SAT scores and generally stagnant results from high schools on federal tests and other measures reflect a troubling shortcoming of education-reform efforts. The test results show that gains in reading and math in elementary grades haven’t led to broad improvement in high schools, experts say. That means several hundred thousand teenagers, especially those who grew up poor, are leaving school every year unready for college.

“Why is education reform hitting a wall in high school?” asked Michael J. Petrilli, president of the Thomas B. Fordham Institute, a think tank. “You see this in all kinds of evidence. Kids don’t make a whole lot of gains once they’re in high school. It certainly should raise an alarm.”

“It is difficult to pinpoint a reason for the decline in SAT scores,” Anderson said, making a statement which fell just short of a lie. In fact, the SATs can’t be used for making these kinds of comparisons, a basic fact Petrilli noted in a blog post later that day. But even as Anderson pimped some very familiar gloom with a bungled front-page report, Petrilli seemed to support his overall gloomy assessment.

Have high school seniors been on the skids? Is math achievement on the decline? Anderson seemed to be saying just that, and his elite “educational expert” seemed to be agreeing.

“You see this in all kinds of evidence,” the expert reliably said. As part of this scripted recitation, Anderson made a gloomy reference to those “federal tests,” which were said to show “generally stagnant results.”

To what “federal tests” was Anderson referring? He never explained, but the answer is clear—he was referring to the National Assessment of Educational Progress (NAEP), the widely-praised “gold standard” of domestic educational testing.

Are high school seniors stagnating on the NAEP, as Anderson gloomily said? Not exactly, no! No, it seems that they are not, as we will show you below.

Beyond that, a word of warning: as with the SATs, so too with the Grade 12 NAEP! There’s a basic reason why it’s hard to make reliable comparisons over time with NAEP testing in Grade 12—a basic problem which doesn’t obtain with NAEP testing in Grade 4 and Grade 8.

Petrilli identified that problem too, in his blog post later that day. But in the Washington Post, it was all gloom and doom, just as the funders prefer!

Tomorrow, we’re going to take you through Petrilli’s full blog post. In our view, it’s an intellectual and/or moral disgrace—the kind of work which helps us see how utterly faux our educational discourse actually is.

For today, let’s set ourselves a simpler task. Let’s look at Grade 12 math scores on those “federal tests” over the past ten years. How “stagnant” have those math results actually been?

People, let’s break every rule in the book—let’s take a look at the record! First, a bit of background:

The NAEP conducts two major parallel studies. It conducts the so-called “Main NAEP,” which test students in Grades 4, 8 and 12.

(Drop-outs don’t get tested.)

The NAEP also conducts the so-called “Long-Term Trend Study,” which tests 9-year-old students, 13-year-old students and 17-year-old students, no matter what grade they’re in.

(Once again, drop-outs don’t get tested. This becomes an important fact in Grade 12 and 17-year-old testing, a point we’ll discuss tomorrow.)

For today, let’s ignore the statistical problem caused by high school drop-outs. Let’s stick to a simpler question: How have actual high school seniors been doing on those federal tests?

Another small bit of background:

The Grade 12 math test on the Main NAEP was revised in 2005. In the best direct comparison we currently have, we can compare math scores attained in 2005 with math scores attained in 2013. (Results from the 2015 testing aren't available yet.)

How have high school seniors been doing in math? Over the most recent eight-year period we can review, these gains in average scores have been recorded:

Gains in average scores, 2005-2013
Main NAEP, Grade 12 math
National public schools
White students: 4.32 points
Black students: 5.24 points
Hispanic students: 7.67 points
Asian-American students: 11.08 points

Those are the actual score gains recorded by the nation’s actual high school seniors during that actual eight-year period. And yes, the NAEP tests representative samples of high school seniors, unlike the SAT!

An obvious question arises:

Are those significant score gains? Or should those gains be viewed as a type of “stagnation?”

We’d love to see some actual journalists and educational experts address that important question. For today, we’ll cite the familiar, rough rule of thumb we’ve cited many times in the past.

When reporting results from the NAEP, journalists frequently apply a rule of thumb. We’ve always referred to it as a very rough rule of thumb, although journalists will sometimes apply it quite literally when it produces gloomy results about the daunting size of our “achievement gaps.”

Journalists almost never apply the rule of thumb to students’ score gains. In fact, they virtually never discuss such gains, except to deny that any such gains have occurred. But according to this very rough rule of thumb, ten points on the NAEP scale is often compared to one academic year.

Applying that very rough rule of thumb, those look like substantial score gains on the Grade 12 NAEP. That’s especially true when we consider the statistical role played by high school drop-outs, which we’ll discuss tomorrow.

(To see Petrilli discuss it in his blog post, just click here.)

Good grief! If we apply that very rough rule of thumb, black high school seniors gained more than half a year in average math achievement over an eight-year period. The gains by Hispanic and Asian-American high school seniors were even larger.

Unless we’re being completely irrational, that would count as rapid growth. Until you read the Washington Post! Or until you hear the familiar words of one of our well-funded “experts.”

Why would Anderson say that scores have stagnated on those “federal tests?” There’s a shaky answer to that question, which we’ll explore tomorrow.

But when we do, we’ll look at the full blog post Petrilli offered on the same day he was quoted in the Post. In our view, his blog post is a work of intellectual and/or moral squalor.

Michael Moore explained this aspect of our culture back in 2003. “We live in fictitious times,” he said. “We live in the time where we have fictitious election results that elect a fictitious president.”

Those election results were stone-cold sober compared to our education reporting. We know of no other part of our public discourse where the “fictitions” are so familiar and so dominant.

These fictitions are shouted in our ears by our journalists and our “experts.” They dominated what appeared in the Washington Post that day.

On Thursday, we’ll return to those professors at Penn and their peculiar study. Tomorrow, let’s review what the expert said.

We think his blog post was a disgrace. But then, what else is new?

Tomorrow: Pure squalor—a scripted disgrace

Where the data come from: The National Center for Education Statistics produces tons of data from the NAEP. Unfortunately, there is no known way to make our “journalists” review, report or acknowledge the existence of such data.

Dearest darlings, it just isn’t done! In our education discourse, it’s narrative all the way down!

Back to the facts-in-themselves! If you want to examine Grade 12 math results from the Main NAEP, you’ll have to do these things:

For starters, just click this. Then, click on MAIN NDE (Main NAEP Data Explorer).

After that, click on “I agree to the terms above.”

At that point, you’re on your own. But all the data are there