EXPERTS, PROFESSORS AND JOURNOS: Good grief!

WEDNESDAY, SEPTEMBER 16, 2015

Part 3—“Educational expert” at work:
Are high school seniors doing less well in math? Might they be doing better?

Such questions are hard to answer. The SATs cannot be used to answer such questions, as everyone secretly knows. (For evidence, see below.)

If the SATs can't be used, how about the National Assessment for Educational Progress (NAEP), the widely-praised “gold standard” of domestic educational testing? Can’t we start to answer those questions by looking at NAEP results?

Presumably yes, as we noted yesterday and as we’ll see below. That said, there’s a statistical problem dogging the NAEP when we reach the Grade 12 level.

Ironically, this statistical problem is connected to our declining drop-out rates! For the record, everyone understands this statistical problem. In fact, a leading “educational expert” recently explained it in a blog post.

For now, this expert will go unnamed. For now, don’t worry about what he’s explaining here. His basic point is clear:
UNNAMED EDUCATIONAL EXPERT (9/3/15): One explanation could be America’s rising graduation rate. Students who would have previously dropped out are now staying in school and remaining in the NAEP sample, thereby dragging down the scores. That sounds plausible to me...
Everyone understands this statistical problem, even educational experts! Here’s the way it lays:

The nation’s drop-out rate has been declining in recent decades and years. In theory, that’s a good thing! But it introduces a bit of statistical mud into Grade 12 NAEP scores.

Lower-achieving kids who once dropped out are now staying in school! In theory, that’s a good thing. But as this educational expert noted in that recent blog post, this probably tends to depress average scores over the course of the years.

At the Grade 12 level, average NAEP scores may get dragged down, over time, by the drop in drop-out rates. The unnamed educational expert explained this possibility rather clearly.

You’ll want to keep that possible factor in mind when you revisit the data we showed you in yesterday’s post. Truth to tell, average math scores have been on the rise in the most recent eight-year span for which we have Grade 12 data from the NAEP:
Gains in average scores, 2005-2013
Main NAEP, Grade 12 math
National public schools

White students: 4.32 points
Black students: 5.24 points
Hispanic students: 7.67 points
Asian-American students: 11.08 points
Just to complete the record, American Indian/Native Alaskan students recorded a score gain of 9.48 points over that eight-year span.

As we noted yesterday, those are the actual score gains recorded by our actual high school seniors on our most reliable tests. Those gains were recorded in spite of the possibility that average scores are being depressed by declining drop-out rates.

As we noted yesterday, those score gains seem to be rather substantial, as judged by standard rules of thumb. Why then did we read what follows in the Washington Post? We read this on the famous newspaper’s front page, no less:
ANDERSON (9/3/15): Scores on the SAT have sunk to the lowest level since the college admission test was overhauled in 2005, adding to worries about student performance in the nation’s high schools.

[...]

The steady decline in SAT scores and generally stagnant results from high schools on federal tests and other measures reflect a troubling shortcoming of education-reform efforts. The test results show that gains in reading and math in elementary grades haven’t led to broad improvement in high schools, experts say. That means several hundred thousand teenagers, especially those who grew up poor, are leaving school every year unready for college.

“Why is education reform hitting a wall in high school?” asked Michael J. Petrilli, president of the Thomas B. Fordham Institute, a think tank. “You see this in all kinds of evidence. Kids don’t make a whole lot of gains once they’re in high school. It certainly should raise an alarm.”
As we’ll show you below, everyone knows that the SATs can’t be used for this purpose. Meanwhile, the “federal tests” to which Anderson refers are those administered by the NAEP, on which high school seniors seem to have shown substantial gains in math in recent years.

Can we talk? The Washington Post has played this apparent game for a very long time now. Until recently, cynics said the Post did this because of its corporate involvement in the testing industry through its Kaplan, Inc. affiliate, which provided the Washington Post Company with the bulk of its profits.

Now that the Post has changed hands, this explanation seems to fail—but the gloomy narrative lives! Test scores are always down when you’re reading the Washington Post—or, at the very best, scores are allowed to be “stagnant.” And there was Petrilli, a go-to “educational expert,” helping spread the gloom and doom early in this front-page news report.

To appearances, Petrilli had spoken in support of Anderson’s gloomy perspective. You can see the stagnation in high school achievement “in all kinds of evidence,” the educational expert said.

Anderson cited the SATs and those unnamed “federal tests” which were, in fact, the NAEP. Petrilli seemed to speak in support of this gloomy assessment.

Had Petrilli been misquoted? Had his words been taken out of context? That very day, this billionaire-funded “educational expert” took to the Internet to state his views about the stagnation in a bit more detail.

We think his blog post was a disgrace—an act of moral and/or intellectual squalor. On the brighter side, we also think it might help us see the soul of the modern elite “educational expert.”

What did the expert say in his post? Amazingly, he started out like this:
PETRILLI (9/3/15): The latest SAT scores are out today, and as I remarked to Nick Anderson at the Washington Post, education reform appears to be hitting a wall in high school.

In truth, we already knew this. The SATs aren’t even the best gauge—not all students take them, and those who do are hardly representative.
Students who take the SATs “are hardly representative?” Plainly, this is true—and the demographic blend of the students who take the tests has been changing in substantial ways, in ways which everyone knows will drag average scores down.

Students who take the SATs are “hardly representative?” If so, why was Petrilli quoted in the Post supporting Anderson’s gloom and doom about those declining scores?

Alas! As the expert’s blog post continued, the analysts started to wail:
PETRILLI (continuing directly): But a variety of sources show much the same thing. Twelfth-grade NAEP: Flat. Long-term NAEP for seventeen-year-olds: Flat. ACT scores: Flat. Percentage of college-ready graduates: Flat.
For today, let’s note two points in that paragraph. As we do, we’ll marvel at the type of work our “experts” routinely perform.

First point: Based on the data we’ve posted above, does it look to you like scores on the “twelfth grade NAEP” have been “flat?” It doesn’t look that way to us! But as we’ve shown you again and again, all roads must lead to that conclusion when mainstream figures talk about test scores!

Are twelfth-grade NAEP scores flat? Alas! In support of that peculiar claim, Petrilli didn’t link to any actual NAEP scores! Instead, he linked to a 2014 news report—a news report by a Christian Science Monitor “staff writer.”

How bad can it get when “staff writers” at our newspapers attempt to report test scores? We’ll be exploring that question as our series continues—and you’ll see some awful results.

Warning! Most “staff writers” lack the first clue in this semi-technical area. But it isn’t just our journalists who make a joke of our discourse.

Our professors and experts make a mess too; so do corporate spokesmen and government officials. Good grief! In the confused and confusing news report to which Petrilli chose to link, this pitiful passage appears:
PAULSON (5/7/14): “Despite the highest high school graduation rate in our history, and despite growth in student achievement over time in elementary school and middle school, student achievement at the high school level has been flat in recent years. Just as troubling, achievement gaps among ethnic groups have not narrowed,” said Education Secretary Arne Duncan in a statement, responding to the NAEP scores.
How pathetic can Arne Duncan get? In the first sentence quoted above, he seems to think that the rise in graduation rate should be driving average NAEP scores up!

Petrilli seems to know better. In fact, he’s the unnamed educational expert who explained, in his recent blog post, that the rising graduation rate tends to bring average scores down on the NAEP!

Duncan made a peculiar statement. An unskilled staff writer typed it up; Petrilli linked to her anyway. And by the way:

The math score gains we’ve posted above are among the scores to which Duncan was referring that day! Judged by normal rules of thumb, those score gains seemed substantial, even before we factored in declining drop-out rates.

As always, though, the tools of our billionaire “reformers” refused to report this basic fact. Arne Duncan reviewed the data and said the scores were “flat.”

That news report in the CSM was confused, confusing—incompetent. Almost always, that’s what happens when “staff writers” are asked to report test results in this specialized, semi-technical field.

“Twelfth-grade NAEP: Flat?”

In even a slightly rational world, Petrilli would be canned, right now, for posting that assessment. But good God! Those four words are straight outta Einstein compared to this pathetic emission from our “educational expert:”

“ACT scores: Flat.”

Good lord, how the analysts howled! Let’s understand why they screamed when they read those words:

In his blog post, Petrilli had already explained a bone-simple fact—SAT scores cannot be reliably used for making these types of assessments. But then, to support the mandated gloomy assessment, he noted that ACT scores are flat, ignoring the fact that ACT scores can’t be used for this purpose either!

Good God! As with the SAT, so with the ACT—students who take the ACT are “hardly representative!” And as with the SAT, so too with the ACT—the program’s demographic blend is changing every year, in ways which are guaranteed to lower average scores.

Everyone understands these points—and yet, the expert wrote what he did. At this point, you’re left with a choice:

You can decide that Michael Petrilli is a hopeless incompetent. Good grief! Who else would note the inutility of the SAT, then turn instead to the ACT?

Second choice: You can decide that Petrilli’s a hack, a man who’s been commissioned by billionaire funders to deliver a certain message.

“The mail must go through,” the federal government once said. Today, so must the narratives of the billionaire funders of “education reform!”

Test scores must be at their lowest point ever. At best, test scores are allowed to be “stagnant,” which almost sounds worse. No other facts need apply!

Tomorrow, we’ll look at the rest of Petrilli’s blog post. But are these “experts” really for real? Or are they the billionaires’ puppets?

We’ve posted the actual score gains in math recorded by our high school seniors.

Do Grade 12 math scores seem “flat” to you? Why did the expert say that?

Tomorrow: More from our educational expert

Still coming: Those two professors from Penn

26 comments:

  1. Which commenter, Bobfan or troll, will be the first to open up today's edition of the Great Very Rough Rule of Thumb debate?

    Like the War on Gore and the deep concern over the paychecks of MSNBC hosts, it never gets old.

    ReplyDelete
    Replies
    1. Not enough people want to engage in meaningful discourse about why are teacher are so ratty and their unions so infernal.

      Delete
    2. Fear not! Bob will take a look at his dwindling click rate when he writes about phenomenal gains and rules of thumb, and soon we will be treated to a series on the Myth of the Gender Wage Gap, or How Universities Can't Tell the Difference between Flirting and Rape.

      And failing that, he will reopen the completely justifiable homicides of Trayvon Martin and Michael Brown, as he ponders why all these black people are still mad since we fixed everything 50 years ago.

      That'll bring 'em back. And once they are here, then he can re-examine the War on Gore, while lecturing "liberals" not to get too upset about those weapons of mass destruction that still could turn up any day now.

      Delete
    3. Bob is too busy counting words in the offerings of our morally disgraceful press to count his own clicks.

      Delete
    4. 12:03, 12:59 and 2:02: You sure put a lot of energy into saying nothing of substance to counter what Somerby is saying. In the law, silence is often an admission. So is changing the subject and an ad hominem attack.

      Delete
    5. So I take it you agree with 1:55 then, urban legend?

      Delete
  2. Despite the fantastic reduction in drop out rates* the thumb measures** have improved from "very rough" to "normal" in the course of a single blog!

    * Drop out rates can be measured in a variety of ways, hence we have no Gold Standard Drop out measure.

    ** Somerby measures only public school test scores for his normal thumb measurements. In private schools Nuns use rulers to whack thumbs not thumbs to measure test scores. This may explain why the rough thumbs have returned to normal by a student's senior year.

    ReplyDelete
    Replies
    1. Nitpicking Somerby's adjectives is a troll's game.

      Delete
    2. Highlighting Bob's careful use of adjectives to defend him is a Bobfan game...until he changes adjectives.

      Delete
    3. Explaining Your Flawed "Thought" ProcessSeptember 16, 2015 at 1:42 PM

      "very rough" and "normal" are not mutually exclusive.

      Yer welcome.

      Delete
    4. Neither are "dumb" and "liberal" if you are a lazy Somerby reader of dubious morals whom nobody likes.

      Delete
  3. "Now that the Post has changed hands [to Jeff Bezos], this explanation seems to fail."

    From 8/6/13 dianeravich.net:

    The Bezos Family Foundation — whose board includes Bezos, his parents and other family members — gave more than $11 million in 2011 to an array of national organizations such as Teach for America, Stand for Children and the KIPP Foundation, according to tax filings. The foundation also gave grants to scores of individual schools around the country as well as several charter school chains, including Uncommon Schools, which operates schools in New York and Massachusetts.

    Bezos’s parents, Mike and Jackie, were active in a fierce battle last year to allow the creation of public charter schools in Washington state. Washington had been one of a handful of states that did not permit charters, which are publicly funded schools that are privately run and largely without unions. Teachers unions opposed the ballot measure, which narrowly passed with financial backing from Mike and Jackie Bezos as well as Microsoft founder Bill Gates and Netflix founder Reed Hastings.”

    ReplyDelete
    Replies
    1. Thank you for pointing this out.

      Delete
    2. Amazing that Somerby would miss that. Thank heaven for Diane Ravitch.

      Delete
    3. You miss the point: Bob stated that it was the Post's ownership of Kaplan that makes that explanation fail. Your reference to the Bezos family is irrelevant.
      Perhaps you should stick to your Dick and Jane readers.

      Delete
    4. I wonder who the cynic is in this formula and whether Somerby's use of the word was ironic in support of those who have pointed out the Post's bias, or if he is taking a tiny glee in the fact that the "explanation seems to fail" in preference of another explanation.

      Not to toot my own horn, or Spanky's for being on the ball here, or Ravitch's for bothering to publicize it, but come on, Mr. Bob! This's more than a lack of intellectual curiosity your betraying. One might be tempted to call you a cynic.

      Delete
    5. " ... but the gloomy narrative lives."

      Horace,
      You need to learn how to read - period.

      Another explanation for continuation of the Post's gloomy education slant could be Bezos' involvement in for-profit education.

      Which one are you, Dick or Jane?

      Delete
  4. The problem may be that Petrilli has not disaggregated the scores.

    ReplyDelete
    Replies
    1. Your problem may be that you read Somerby but don't follow his links.

      "Or maybe it’s Simpson’s Paradox at work. That would suggest that all racial groups are doing better, but because lower-scoring Latinos are replacing whites over time, our overall scores are declining. Yet here are the long-term trends by race:

      (A chart of NAEP disaggregated scores is displayed by Petrilli)

      Looks pretty darn flat to my eyes."

      Delete
  5. Douchebag Troll Parade ... ForeverSeptember 16, 2015 at 12:55 PM

    Education reporting is fine. The BIGGER problem? Somerby.

    ReplyDelete
    Replies
    1. The nation’s drop-out rate has been declining in recent decades and years. In theory, that’s a good thing!

      Delete
    2. Sure, using SAT and ACT scores to assess the education system is just fine reporting.

      Delete
  6. As Bob points out, Arne Duncan seems not to have focused on the difference between these two questions:
    1. Are high school seniors doing less well in math?
    2. Are 18-year olds doing less well in math?

    There's lots of discussion about #1, but I contend that #2 is more important. People don't cease to be human beings just because they don't finish high school. They still have to apply for jobs, pay taxes, make decisions about spending, investment and borrowing options, etc. Level of math capability matters for everyone.

    BTW I wonder what the drop-out rate is in other countries when we compare educational results.

    ReplyDelete
  7. OMB (Multiple Question Choices from the OTB)

    Part 1: Lyin and Bloggin Test Scores, Oh My.

    An interesting series of posts from BOB with a variety of interesting questions. Let us review some in chronological order:

    9/10 "Are 17-year-old American students doing less well in math?

    9/11 "Are seniors in American high schools doing less well in reading and math?

    9/15 "Have high school seniors been doing less well in math in recent years?

    9/16 "Are high school seniors doing less well in math?

    We do not repeat these questions to mock the oft noted repetitiveness of the One True Bob. We salute the subtle but critical changes these evolving questions represent.

    But we have a question of our own.

    Has anyone BOB has quoted in this series said American high school seniors are doing less well in math? Or is BOB setting up his latest version of Conquering the Miracle Polish Strawman? Or was it Strawman Miraculously Conquers the Poles? We forget.

    We do remember the changes in the questions. First it was 17 year olds and math. Then it was seniors in math and reading, Next it switched to seniors in math in recent years.
    Finally he ended with today's question, which opens a long post which concludes "Do Grade 12 math scores seem “flat” to you? Why did the expert say that?"

    Bob tells us we have two choices about this "expert." He is "a hopeless incompetent" or "Petrilli’s a hack, a man who’s been commissioned by billionaire funders to deliver a certain message."

    Well, there is another choice. Since Petrilli never said "Grade 12 Math Scores Are Flat" we'd say BOB is a competent, though impoverished, liar.

    Why did BOB keep refining a question which seemed, at casual glance, to be just another of those endless repetitions in the best BOBstyle?

    Because he had to do so in order to narrow down the test results needed to prove somebody was lying about an assertion that was never made. At least not by anybody BOB quoted in this series.

    Which brings us to the Poles. Not too long ago, BOB wanted to prove there was no Polish "miracle" on PISA test scores as part of an effort to refute any and everything said by a certain Ms. Ripley, who committed the unpardonable sin of getting a full book written and published to some critical acclaim. He went to elaborate lengths to manipulate test scores and test score gains, even to the point of combining gains from scores on different subject matter tests, to try and prove American students had not been outdone by Polish kids on PISA tests. This was a claim made by reviewers of Ms. Ripley's book.

    That old BOB is back. This time among his tricks are to disappear tests until he gets the ones which fit his meme, accuse his opponents of saying things they did not, and resting his case on one statement made by one person which is based on the very type of analysis used by BOB himself in "deflating" the Polish miracle.

    In Part 2 we'll show you how these things were done. Hint: If you are still reading, one of the things that disappeared was "reading."

    ReplyDelete
  8. My husband cheated on me for Almost three years. he ignore me for several months and left me with nothing, but i am happy today that Chief Nwaluta brought my husband back, I am so happy, Now my husband is all mine again. I can now say I'm happy again. Great spell from Chief Nwaluta, Chief Nwaluta is genuine. I truly believe in him and his spells. he is a professional. , My name is Sandra Gault and I live in Bryan Texas, My husband and i got married for more than 11 years and have gotten two kids. thing were going well with us and we are always happy. until one day my husband started to behave in a way i could not understand, i was very confused by the way he treat me and the kids. later that month he did not come home again and he called me that he want a divorce, i asked him what have i done wrong to deserve this from him, all he was saying is that he want a divorce that he hate me and do not want to see me again in his life, i was mad and also frustrated do not know what to do,i was sick for more than 2 weeks because of the divorce. i love him so much he was everything to me without him my life is incomplete. i told my sister and she told me to contact a spell caster, i never believe in all this spell casting of a thing. i just want to try if something will come out of it. i contacted Chief Nwaluta for the return of my husband to me, they told me that my husband have been taken by another woman, that she cast a spell on him that is why he hate me and also want us to divorce. then they told me that they have to cast a spell on him that will make him return to me and the kids, they casted the spell and after 1 week my husband called me and he told me that i should forgive him, he started to apologize on phone and said that he still love me that he did not know what happen to him that he left me. it was the spell that he Chief Nwaluta casted on him that make him come back to me today,me and my family are now happy again today. thank you Chief Nwaluta for what you have done for me i would have been nothing today if not for your great spell. i want you my friends who are passing through all this kind of love problem of getting back their husband, wife , or ex boyfriend and girlfriend to contact Chief Nwaluta,if you need his help you can contact him through his private E-mail:{ Nwalutaspelltemple@gmail.com } or you can contact him through his Web site :http://nwalutaspelltemple.blogspot.com. Thanks you Chief Nwaluta.

    ReplyDelete

  9. All thanks to the great doctor that help me to get back my boyfriend I thank you so much because you man of your words than anything he said will be done is a very good man I want to use opportunity thank Dr Ben and God shall bless him for getting my boyfriend back he is back to me now I didn't believe this at first until I put effort now my boyfriend is back I thank Dr Ben for the great work He has done for me I want him to keep doing this for other people are so that you can do it for me to thank you Dr Ben. And you can contact him at email DrBenspellcaster@gmail.com or whatsapp number +2348151642717. Now I believe you are the great spell caster Dr Ben.

    ReplyDelete