EXPERTS, PROFESSORS AND JOURNOS: Good grief!

WEDNESDAY, SEPTEMBER 16, 2015

Part 3—“Educational expert” at work: Are high school seniors doing less well in math? Might they be doing better?

Such questions are hard to answer. The SATs cannot be used to answer such questions, as everyone secretly knows. (For evidence, see below.)

If the SATs can't be used, how about the National Assessment for Educational Progress (NAEP), the widely-praised “gold standard” of domestic educational testing? Can’t we start to answer those questions by looking at NAEP results?

Presumably yes, as we noted yesterday and as we’ll see below. That said, there’s a statistical problem dogging the NAEP when we reach the Grade 12 level.

Ironically, this statistical problem is connected to our declining drop-out rates! For the record, everyone understands this statistical problem. In fact, a leading “educational expert” recently explained it in a blog post.

For now, this expert will go unnamed. For now, don’t worry about what he’s explaining here. His basic point is clear:

UNNAMED EDUCATIONAL EXPERT (9/3/15): One explanation could be America’s rising graduation rate. Students who would have previously dropped out are now staying in school and remaining in the NAEP sample, thereby dragging down the scores. That sounds plausible to me...

Everyone understands this statistical problem, even educational experts! Here’s the way it lays:

The nation’s drop-out rate has been declining in recent decades and years. In theory, that’s a good thing! But it introduces a bit of statistical mud into Grade 12 NAEP scores.

Lower-achieving kids who once dropped out are now staying in school! In theory, that’s a good thing. But as this educational expert noted in that recent blog post, this probably tends to depress average scores over the course of the years.

At the Grade 12 level, average NAEP scores may get dragged down, over time, by the drop in drop-out rates. The unnamed educational expert explained this possibility rather clearly.

You’ll want to keep that possible factor in mind when you revisit the data we showed you in yesterday’s post. Truth to tell, average math scores have been on the rise in the most recent eight-year span for which we have Grade 12 data from the NAEP:

Gains in average scores, 2005-2013
Main NAEP, Grade 12 math
National public schools
White students: 4.32 points
Black students: 5.24 points
Hispanic students: 7.67 points
Asian-American students: 11.08 points

Just to complete the record, American Indian/Native Alaskan students recorded a score gain of 9.48 points over that eight-year span.

As we noted yesterday, those are the actual score gains recorded by our actual high school seniors on our most reliable tests. Those gains were recorded in spite of the possibility that average scores are being depressed by declining drop-out rates.

As we noted yesterday, those score gains seem to be rather substantial, as judged by standard rules of thumb. Why then did we read what follows in the Washington Post? We read this on the famous newspaper’s front page, no less:

ANDERSON (9/3/15): Scores on the SAT have sunk to the lowest level since the college admission test was overhauled in 2005, adding to worries about student performance in the nation’s high schools.

[...]

The steady decline in SAT scores and generally stagnant results from high schools on federal tests and other measures reflect a troubling shortcoming of education-reform efforts. The test results show that gains in reading and math in elementary grades haven’t led to broad improvement in high schools, experts say. That means several hundred thousand teenagers, especially those who grew up poor, are leaving school every year unready for college.

“Why is education reform hitting a wall in high school?” asked Michael J. Petrilli, president of the Thomas B. Fordham Institute, a think tank. “You see this in all kinds of evidence. Kids don’t make a whole lot of gains once they’re in high school. It certainly should raise an alarm.”

As we’ll show you below, everyone knows that the SATs can’t be used for this purpose. Meanwhile, the “federal tests” to which Anderson refers are those administered by the NAEP, on which high school seniors seem to have shown substantial gains in math in recent years.

Can we talk? The Washington Post has played this apparent game for a very long time now. Until recently, cynics said the Post did this because of its corporate involvement in the testing industry through its Kaplan, Inc. affiliate, which provided the Washington Post Company with the bulk of its profits.

Now that the Post has changed hands, this explanation seems to fail—but the gloomy narrative lives! Test scores are always down when you’re reading the Washington Post—or, at the very best, scores are allowed to be “stagnant.” And there was Petrilli, a go-to “educational expert,” helping spread the gloom and doom early in this front-page news report.

To appearances, Petrilli had spoken in support of Anderson’s gloomy perspective. You can see the stagnation in high school achievement “in all kinds of evidence,” the educational expert said.

Anderson cited the SATs and those unnamed “federal tests” which were, in fact, the NAEP. Petrilli seemed to speak in support of this gloomy assessment.

Had Petrilli been misquoted? Had his words been taken out of context? That very day, this billionaire-funded “educational expert” took to the Internet to state his views about the stagnation in a bit more detail.

We think his blog post was a disgrace—an act of moral and/or intellectual squalor. On the brighter side, we also think it might help us see the soul of the modern elite “educational expert.”

What did the expert say in his post? Amazingly, he started out like this:

PETRILLI (9/3/15): The latest SAT scores are out today, and as I remarked to Nick Anderson at the Washington Post, education reform appears to be hitting a wall in high school.

In truth, we already knew this. The SATs aren’t even the best gauge—not all students take them, and those who do are hardly representative.

Students who take the SATs “are hardly representative?” Plainly, this is true—and the demographic blend of the students who take the tests has been changing in substantial ways, in ways which everyone knows will drag average scores down.

Students who take the SATs are “hardly representative?” If so, why was Petrilli quoted in the Post supporting Anderson’s gloom and doom about those declining scores?

Alas! As the expert’s blog post continued, the analysts started to wail:

PETRILLI (continuing directly): But a variety of sources show much the same thing. Twelfth-grade NAEP: Flat. Long-term NAEP for seventeen-year-olds: Flat. ACT scores: Flat. Percentage of college-ready graduates: Flat.

For today, let’s note two points in that paragraph. As we do, we’ll marvel at the type of work our “experts” routinely perform.

First point: Based on the data we’ve posted above, does it look to you like scores on the “twelfth grade NAEP” have been “flat?” It doesn’t look that way to us! But as we’ve shown you again and again, all roads must lead to that conclusion when mainstream figures talk about test scores!

Are twelfth-grade NAEP scores flat? Alas! In support of that peculiar claim, Petrilli didn’t link to any actual NAEP scores! Instead, he linked to a 2014 news report—a news report by a Christian Science Monitor “staff writer.”

How bad can it get when “staff writers” at our newspapers attempt to report test scores? We’ll be exploring that question as our series continues—and you’ll see some awful results.

Warning! Most “staff writers” lack the first clue in this semi-technical area. But it isn’t just our journalists who make a joke of our discourse.

Our professors and experts make a mess too; so do corporate spokesmen and government officials. Good grief! In the confused and confusing news report to which Petrilli chose to link, this pitiful passage appears:

PAULSON (5/7/14): “Despite the highest high school graduation rate in our history, and despite growth in student achievement over time in elementary school and middle school, student achievement at the high school level has been flat in recent years. Just as troubling, achievement gaps among ethnic groups have not narrowed,” said Education Secretary Arne Duncan in a statement, responding to the NAEP scores.

How pathetic can Arne Duncan get? In the first sentence quoted above, he seems to think that the rise in graduation rate should be driving average NAEP scores up!

Petrilli seems to know better. In fact, he’s the unnamed educational expert who explained, in his recent blog post, that the rising graduation rate tends to bring average scores down on the NAEP!

Duncan made a peculiar statement. An unskilled staff writer typed it up; Petrilli linked to her anyway. And by the way:

The math score gains we’ve posted above are among the scores to which Duncan was referring that day! Judged by normal rules of thumb, those score gains seemed substantial, even before we factored in declining drop-out rates.

As always, though, the tools of our billionaire “reformers” refused to report this basic fact. Arne Duncan reviewed the data and said the scores were “flat.”

That news report in the CSM was confused, confusing—incompetent. Almost always, that’s what happens when “staff writers” are asked to report test results in this specialized, semi-technical field.

“Twelfth-grade NAEP: Flat?”

In even a slightly rational world, Petrilli would be canned, right now, for posting that assessment. But good God! Those four words are straight outta Einstein compared to this pathetic emission from our “educational expert:”

“ACT scores: Flat.”

Good lord, how the analysts howled! Let’s understand why they screamed when they read those words:

In his blog post, Petrilli had already explained a bone-simple fact—SAT scores cannot be reliably used for making these types of assessments. But then, to support the mandated gloomy assessment, he noted that ACT scores are flat, ignoring the fact that ACT scores can’t be used for this purpose either!

Good God! As with the SAT, so with the ACT—students who take the ACT are “hardly representative!” And as with the SAT, so too with the ACT—the program’s demographic blend is changing every year, in ways which are guaranteed to lower average scores.

Everyone understands these points—and yet, the expert wrote what he did. At this point, you’re left with a choice:

You can decide that Michael Petrilli is a hopeless incompetent. Good grief! Who else would note the inutility of the SAT, then turn instead to the ACT?

Second choice: You can decide that Petrilli’s a hack, a man who’s been commissioned by billionaire funders to deliver a certain message.

“The mail must go through,” the federal government once said. Today, so must the narratives of the billionaire funders of “education reform!”

Test scores must be at their lowest point ever. At best, test scores are allowed to be “stagnant,” which almost sounds worse. No other facts need apply!

Tomorrow, we’ll look at the rest of Petrilli’s blog post. But are these “experts” really for real? Or are they the billionaires’ puppets?

We’ve posted the actual score gains in math recorded by our high school seniors.

Do Grade 12 math scores seem “flat” to you? Why did the expert say that?

Tomorrow: More from our educational expert

Still coming: Those two professors from Penn