SOURCES OF PARALYSIS: The New York Times fails the latest tests!

THURSDAY, AUGUST 8, 2013

Interlude—Death in life: Intellectual paralysis grips the heart of our nation’s upper-end “press corps.”

You might think of this state as “death-in-life.” The New York Times announces this paralytic condition with the headline which banners page A19 in today’s hard-copy Washington Edition:

“Scores on Math and English Tests Plummet After State Adopts New Standards”

The headline serves to create excitement, a condition typically sought by the dead-in-life. They obsess about sex to make dead blood flow. They seek the enervation of “scandal.”

Sadly, this news report appears at the top of page A1—the front page!—in this morning’s New York Edition. In that location, it is accompanied by a thrilling trio of headlines.

That headline creates excitement! It also creates tremendous confusion, a trademark of the modern Times. So does the excited start to the newspaper’s news report, which appears at the top of page 1 all over the state of New York.

This is intellectual paralysis, an example of death in life:
HERNANDEZ AND GEBELOFF (8/8/13): The number of New York students passing state reading and math exams dropped drastically this year, education officials reported on Wednesday, unsettling parents, principals and teachers and posing new challenges to a national effort to toughen academic standards.

In New York City, 26 percent of students in third through eighth grade passed the tests in English, and 30 percent passed in math, according to the New York State Education Department.

The exams were some of the first in the nation to be aligned with a more rigorous set of standards known as the Common Core, which emphasize deep analysis and creative problem-solving over short answers and memorization. Last year, under an easier test, 47 percent of city students passed in English, and 60 percent in math.

City and state officials spent months trying to steel the public for the grim figures.

But when the results were released, many educators responded with shock that their students measured up so poorly against the new yardsticks of achievement.
Really? “Many educators responded with shock” to these results? If that is true, a question arises: Why aren’t such people fired?

But then, let’s ask a more direct question—why aren’t Hernandez and Gebeloff fired, along with their unnamed editor? We only receive the comments of educators through the filter of their work—and their work is horrendously bad, starting with that banner headline.

The sheer absurdity of their report is obvious in those opening paragraphs, although the scribes, being dead-in-life, work hard to obscure that fact. Let’s review some basic facts they’re reporting, using some of their own words:

Last year, the state of New York administered “an easier test” on a statewide basis. This year, then, the state of New York administered a harder test.

It’s hard to know why anyone would be “shocked” if passing rates on the new, harder test were lower than on the old, easier test. But that’s the condition these Timesmen describe in their thrilling but paralytic report.

Let’s conduct a thought experiment. Imagine this had occurred:

As part of a statewide test of physical fitness, the state of New York has been asking all high school students to take a test in which a high jump bar is set at 4 feet. But last year, the state decided to raise the bar. The bar now stands at 5 feet.

No one but the dead would be “shocked” by a drop in the passing rate. But so what? This is the way the New York Times would report the new results:
Scores on Physical Fitness Test Plummet After State Adopts New Standard

The number of New York students passing state physical fitness exams dropped drastically this year, education officials reported on Wednesday, unsettling parents, principals and teachers and posing new challenges to a national effort to toughen fitness standards.

In New York City, 26 percent of students in ninth through twelfth grade passed the tests, according to the New York State Education Department.

The exams were some of the first in the nation to be aligned with a more rigorous set of standards for physical fitness. Last year, under an easier test, 47 percent of city students passed the tests.

City and state officials spent months trying to steel the public for the grim figures.

But when the results were released, many educators responded with shock that their students measured up so poorly against the new yardsticks of achievement.
Can you imagine the New York Times writing such a report? (Hold on! Don’t answer so quickly!) But that is precisely what the paper has done in the case of these reading and math tests.

If last year’s test was “an easier test,” why would anyone be surprised—let alone “shocked”—to learn that fewer students “passed” the new test this year? Why would any sensible person regard the new figures as “grim?”

Indeed, the mental incompetence of the Times begins with that banner headline, which implies that this year’s passing rates can be compared to last year’s rates in some meaningful way.

But duh! Because the passing rates come from two different tests; because the two tests are not “equivalent;” because there has been no attempt to create a statistical bridge between the two tests—for all these reasons, the passing rates from the two tests can’t be compared in a meaningful way! For that reason, a journalist with even minimal competence would have reworked that headline in this way:

Statewide Passing Rates Drop, As Expected, on More Difficult Tests

There is no mystery in what has occurred. But the Times has worked hard to create one.

The dumbness found all through this report is a marker of intellectual paralysis, of what we’ll call death-in-life. Example: If he has been excerpted fairly, this pol should have his ass kicked all the way through the Holland Tunnel:
HERNANDEZ AND GEBELOFF: William C. Thompson Jr., a Democratic candidate who has been endorsed by the city’s teachers’ union, said the results showed that for years the city had put too much of an emphasis on tests at the expense of deeper learning.

“Let’s be clear: We’ve accomplished nothing,” Mr. Thompson, a former city comptroller, said at a news conference. “Doors of opportunity are closing for families and communities across our city.”
Has New York City “accomplished nothing” in its schools over the past X number of years? There is no way to tell from these results, as any living creature could tell you.

Yet all through their paralytic report, the Timesmen sprinkle silly references to “the dreary numbers,” the “bleak scores,” the “grim figures.” Early on, they quote an educator who is said to be shocked, thus adding to the confusion:
HERNANDEZ AND GEBELOFF: But when the results were released, many educators responded with shock that their students measured up so poorly against the new yardsticks of achievement.

Chrystina Russell, principal of Global Technology Preparatory in East Harlem, said she did not know what she would tell parents, who will receive scores for their children in late August. At her middle school, which serves a large population of students from poor families, 7 percent of students were rated proficient in English, and 10 percent in math. Last year, those numbers were 33 percent and 46 percent, respectively.

“Now we’re going to come out and tell everybody that they’ve accomplished nothing this year and we’ve been pedaling backward?” Ms. Russell said. “It’s depressing.”
If Russell has been fairly quoted, she probably shouldn’t be in a school, though her photo appears in the New York Times, with a caption in which she’s quoted again calling the scores “depressing.” Anyone with an ounce of sense would know what to tell those parents:

The state has adopted a much more difficult set of tests, so many fewer children will “pass.”

No one is “pedaling backward” at Russell’s school, but Russell doesn’t seem to know that. And no one is working harder than the Times to help the confusion spread.

Should the state have adopted these new tests? That is an excellent question, though it lies light-years beyond the intellectual horizons of the dead-in-life the New York Times tends to hire or create.

In our view, there is an obvious logical problem with any testing/reporting program of this type, in which a single “passing” standard is set for an entire grade—with “passing rates,” and no other statistics, presented to the public.

“Norm-referenced” tests like the Iowa Tests approach these matters differently. Each student gets a percentile score, showing how well he or she performed in math or reading as compared to a carefully selected national norm group.

A student may go from the 40th percentile in math one year to the 60th percentile the next. This suggests a degree of improvement, at least as compared to her peers.

But on the pass/fail system employed by the current array of tests, that student may simply be told that she failed to pass the test each year. A student or a school may make significant progress from one year to the next—but depending on where the “passing score” has been set, the statistics may not show this.

(Journalists should report average scores along with passing rates. A school’s average score may improve while its passing rate is unchanged.)

Whatever! Such discussion are light years beyond anything we’ll see in the Times, a newspaper which aims to create excitement, and confusion, from its headlines on down.

It’s very hard for people to see the intellectual paralysis which surrounds us. The New York Times spends millions of dollars each year to make us think that its work is high-minded and smart.

Grasping climbers work quite hard to advance these general misperceptions. (An example tomorrow, from Ezra Klein.) Others refuse to tell the truth about the dead-in-life New York Times and the rest of the dead-in-life press corps.

Krugman won’t tell you; neither will Drum. Dionne would rather wander in the desert. The depth of out intellectual paralysis goes unremarked by all.

Once more, let’s explain:

Last year, high school kids in the state of New York were asked to high jump 4 feet. This year, the state of New York raised the bar to 5 feet.

Fewer kids jumped over the bar. Intellectually paralyzed, dead in life, the New York Times stands amazed by this fact. It photographs an attractive young educator who has responded with shock.

Next post: Death in life via Gail Collins

Tomorrow—part 4: Paralysis via “the left”

12 comments:

  1. Thank you Mr. Somerby for these excellent posts. The repetition helps to keep me reminded that newspaper writers, almost always, have an agenda that does not include informing the public. Sometimes there is substance in the articles, but usually well after the jump.

    Bob is great at deconstructing these pieces, and the lessons learned here help greatly with real comprehension when reading the paper.

    ReplyDelete
  2. Parents may be feeling shock and dismay about these scores. If the teachers do not similarly express shock and dismay, might the parents think the teachers do not care about their kids performance? Isn't this reaction kind of required by the circumstances. I agree that this is a silly reaction to a harder test, but when educators give the appropriate reaction, won't they be seen as explaining away bad results or excusing themselves, which will appear self-serving?

    ReplyDelete
  3. I understood the travesty immediately because of Bob Somerby, and was waiting for this critically important post. What a terrible shame to report in so slanted a manner.

    ReplyDelete
  4. Although I basically agree with this post, I think things are a little more complicated. According to the article, the new exams aren't simply harder. They're qualitatively different from the old ones. They (allegedly) emphasize deep analysis and creative problem-solving, whereas the old ones (allegedly) focused on short answers and memorization. Unlike Bob's analogy, the new exams don't just require doing better on the same material.

    ReplyDelete
    Replies
    1. There is not as clear a distinction between memorization and short answers and deep analysis and creative problem-solving. People like to present these as dichotomies but they are not. You cannot problem solve without knowing facts about a situation, so memory is important to all kinds of questions. "Short answer" refers to a test format not a type of thinking, so you could have a deep analysis question that required a short answer (or a multiple choice answer or an essay). The format for response is independent of the kind of thinking being assessed by a question. There is a knee-jerk reaction that assumes that multiple choice questions only involve "rote" memorization (which refers to the kind of practice or encoding technique not what is being learned) when multiple choice can assess very deep analysis if the questions are written to do so. But, the idea that anyone can do "deep analysis" without knowing facts is wrong.

      When you shift to a new common core and teachers have not "taught to the test" then students will be unfamiliar with the vocabulary and presentation approach and may not recognize even the same content. Memory works best when the context in which ideas are learned closely matches the context in which they are tested -- especially when the wording is the same in both the learned material and on the test. Scores will naturally be higher when teachers have had a chance to present material in the same way as it will be later tested. That doesn't mean students don't know the material or that they are being inappropriately coached. It is just how our memory works -- and why we learn better on the job and in the context in which we will apply our knowledge. In order for students to do well on a completely different test, the material has to be taught from a variety of perspectives, in a variety of tasks and situations, and expressed in a variety of ways -- which takes much more time and effort. With that, new tests will always produce lower results than old ones, regardless of how hard teachers and students have worked to learn the material.

      So, this whole testing effort and the use of test scores to assess teachers and students is kind of a crock.

      Delete
    2. Sorry, typo. Should say "WithOUT that, new tests will always produce lower results..."

      Delete
  5. Er, Bob?

    You're not doing so well on the vocabulary part of the test. "Enervation" doesn't mean what you think it means.

    ReplyDelete
    Replies
    1. Agreed. Few will get that though as it's a common misunderstanding. Enervate and energize are not at all synonymous. Quite the opposite.

      The pedantry hour being over now, though, is there anything to say about the matters of Somerby's post itself -- lest our silence be taken as lack of concern as the trolls would have it.

      Delete
    2. Anon. @11:07.

      Screw the trolls. Somerby knows if you say nothing you are, like Krugman/Drum/Dionnes,
      affirming the validity of this posts with your silence.

      Delete
  6. Isn't the main issue here the new common core? I find myself wondering why students should be expected to do well on material they have not yet been taught, assuming the common core is harder because it requires the students to know more or different things.

    ReplyDelete
    Replies
    1. "I find myself wondering why students should be expected to do well on material they have not yet been taught, assuming the common core is harder because it requires the students to know more or different things."

      I find myself wondering that, too. It is insane to test kids on material they haven't been taught. Adults shouldn't do insane things to kids, even if the adults want a base score to go forward with (which is the only possible "good" reason for testing people on material they haven't been taught.

      Unfortunately, there's a very good political reason to test kids on material they haven't been taught: because they will show "dramatic gains!" on the next round of testing after the material has been presented.

      I'm ashamed of this whole program. It's just wrong to treat 9 year olds like this. If I were a NY parent I would be furious, and not because of the test scores.

      Delete
  7. I am genuinely curious about ethical constraints on NY ( and national) leaders regarding experimenting with children.
    These children derived absolutely no benefit from this test, because the material that would prepare them for the test was rushed out and inadequate, probably to meet a political deadline.
    Children are generally treated differently in public policy, because there's a recognition that they're uniquely vulnerable. Is it right to treat them as experimental subjects in a new testing regime to obtain a "baseline" score?

    ReplyDelete