The Washington Post reports record test scores!

WEDNESDAY, JULY 31, 2013

It happens every spring: Everything we know about Emma Brown is good.

Brown is an education reporter for the Washington Post. She graduated from Stanford in 2000. From 2002 through 2005, she taught seventh grade in Juneau, a city believed to be in Alaska.

In 2009, she got a master’s degree in journalism from Berkeley. This morning, she writes a front-page news report, a type of report we have read ten million times before this.

Her report may be right on the money. But it’s very hard to tell—and it happens every spring!

Brown reports that DC test scores have never been higher. At the start of her report, she quotes the standard exultant claims,which may of course be right:
BROWN (7/31/13): Students in the District’s traditional public schools scored higher than ever on the city’s math and reading tests this year, also posting the largest single-year gain since 2008, according to test results released Tuesday.

The city’s public charter schools, which had higher scores than the traditional system, made their biggest gains since 2009. For the first time, more than half of charter students scored proficient or above in reading on the city tests.

Mayor Vincent C. Gray (D) hailed the results as evidence that the city’s overhaul of public education—including the advent of mayoral control of the schools and the rapid growth of charters—is working.

“I don’t think there’s any doubt we’re on the right path,” Gray said. “We just need to stay the course.”
It happens every spring! The mayor says his overhaul is working, and he may even be right.

That said, there’s a problem with Brown’s report. There’s a watchdog which isn’t barking.

Can we talk? Gains in test scores only have meaning if this year’s tests were the same as last year’s tests, or if they were “equivalent” to them. If these year’s tests were easier than last year’s tests, those score gains don’t mean squat.

Were this year’s tests the same as last year’s? If not, do we know that they were equally difficult? Brown’s report doesn’t say. We think this is somewhat odd.

In recent years, the problem of statewide tests getting easier year to year has come up again and again. Major states have thrown out entire batches of test scores because of this phenomenon.

A few years back, the state of New York, a well-known state, dumped a whole year’s worth of scores because the tests had gotten easier over the years.

The New York Times worked very hard to obscure the way this gong-show occurred, since it reflected badly on Mayor Bloomberg, a billionaire. But this has been a recurrent problem around the nation—unless you read today’s Post.

Maybe there’s some simple answer to our question, and Brown knows what it is. Maybe this year’s “statewide” tests in DC were the same tests from last year!

But the Post gave Brown 1500 words to discuss the latest score gains. Do we know that this year’s tests weren’t easier? How hard would this be to explain?

In recent years, DC was plagued by cheating scandals, a problem education reporters had been avoiding for forty years at that point. Proudly fighting the last war, Brown addresses that situation, as she should have done.

But she never says if this year’s tests are known to be as difficult as last year’s. And sure enough! When she presented the requisite words of caution about score gains, we got the standard piddle:
BROWN: Some experts cautioned that standardized testing doesn’t always tell the whole story. Bob Schaeffer of FairTest said it would be inappropriate to attach much importance to single-year gains. He said scores fluctuate and can reflect demographic changes in schools instead of changes in teaching and learning.

“The exclusive focus on test scores as the measure of educational quality should be replaced with the use of multiple performance measures including rates of graduation, college attendance, post-school employment, criminal justice system involvement, etc.,” Schaeffer said in an e-mail.
That highlighted passage is piddle.

Schaeffer is right—score gains can “reflect demographic changes in schools instead of changes in teaching and learning.” But in this case, the score gains would have to reflect a demographic change in the whole DC student population!

Schaeffer’s participation is required by law in a piece of this type. In this case, his comments are almost wholly irrelevant—and these words of caution are even worse:
BROWN (continuing directly): Sam Chaltain, a journalist who is working on a book about school choice in the city and who is the parent of D.C. charter school students, was similarly skeptical.

“We use test scores as a proxy to make it seem like we actually know whether schools are succeeding or failing,” Chaltain said, adding that the rise in scores in the District leaves much unknown. “We don’t know if kids feel more engaged and motivated. We don’t know if teachers feel more supported and prepared to do their jobs well. We don’t know if families are more or less likely to stay in the system.”
We also don’t know how the students feel about the name of Washington’s NFL team! But that is completely beside the point here, as are the three concerns Chaltain is worrying about.

Were these tests as hard as last year’s tests? If so, how do we know that? This is basic journalistic blocking and tackling, and it’s MIA here.

When we read about Trayvon Martin in the white establishment press, we think about decades of work of this type. Do white liberals actually care about black kids? Or does the establishment press just like to posture and pose?

Plainly, no—we don’t seem to care. If you actually care about a topic, you jealously guard the basic facts. You struggle to get your basic blocking and tackling right.

We have watched the establishment press produce this kind of stumblebum work for more than forty years now. In this case, Brown is fairly new to this beat—but how long has her editor been there?

Urban schools have always been reported in this half-hearted way. At some point, it gets hard to avoid the feeling that these people don’t really care, that our urban schools exist to let assorted public figures take familiar bows.

There may be an answer to our question. We’d like to know what it is.

10 comments:

  1. Bob makes a valid point, I think. Still, it's a strange comparison. Bob has written a lot of posts criticizing people because they ignored student test score improvements. Now he criticizes someone for paying too much attention to student test score improvements.

    ReplyDelete
    Replies
    1. Crown in the MoldAugust 1, 2013 at 6:17 PM

      "Now he criticizes someone for paying too much attention to student test score improvements."

      Somerby: "Were these tests as hard as last year's tests? If so, how do we know that? This is basic journalistic blocking and tackling, and it's MIA here."

      Brown never went there. That's not paying too much attention, that's half-assed reporting.

      Delete
  2. Bob,
    Test scores? True or false?
    From the (lack of) comments it seems obvious that your new found cohorts do not give a flying fig about minority children. A condition seemingly not reserved for "liberals".
    Throw your new fans some:
    Trayvon Martin was responsible for his own death.
    Or: the liberal media is "vile".
    Or: Zimmerman said...
    Or: His parents lawyers are "evil".

    C'mon on Bob. You can do it.

    ReplyDelete
    Replies
    1. This article had only been up on the webpage an hour when you posted this complaint about the lack of comments. It may be true that commenters don't care as much about minority children as they do about Trayvon Martin, but why not give readers a fair chance to read the post before complaining?

      Delete
    2. The premise that bitter debate or its absence proves that people do or don't care about the topic is the mark of poor thinking typical of a troll such as 4:57 Anonymous.

      Perhaps many readers simply agree with this particular posting (if one has been paying attention for long to the topic, one knows that it is true as Somerby says "It happens every spring! The mayor says his overhaul is working, and he may even be right. That said, there’s a problem with Brown’s report. There’s a watchdog which isn’t barking."), and find little to take issue with, therefore the posting isn't the subject of the kind of stupid commentary found on the Martin/Zimmerman posts.

      Howsoever, one thing is absolutely certain -- 4:57 is a typical TDH troll. He hates this little blog and brainlessly spews his venom on whatever it produces.

      Delete
    3. "C'mon Bob, you can do it."

      He did. You missed the tie-in.

      "When we read about Trayvon Martin in the white establishment press, ...." KA-CHING!

      Delete
    4. Anonymous 4:57, I'm right there with ya.

      "This article had only been up on the webpage an hour when you posted this complaint" -- yeah, well now it's been over 24 hours.

      "Perhaps many readers simply agree with this particular posting..." Right, that's the reason. And "perhaps" Mohammed really DID fly up to heaven on that winged horse.

      Delete
  3. I've given up on the test scores after a decade of kids in public schools and a decade of "assessments". The PBS education reporter has a completely different take on the DC scores.
    I believe we have had 4 different scoring systems in Ohio in the last ten years. We are currently following the "letter grade for schools" fad, but reading around I see that this is now discredited after 9 months or something.
    Last year, the held a mandatory assembly where they told the second graders they were getting held back unless they passed their "third grade reading guarantee" test this year. They're 2nd graders, 7 or so, so I bet they've forgotten all about this warning by now. I was talking to some 4th graders after the assembly, and they were concerned about the 2nd graders, so that was nice. They feel sorry for them, being singled out like that. Hell, I feel sorry for them.
    I know these things are mandatory and primarily promoted by politicians, but I think we're all reformed-out here. We wish they'd just go away and leave us alone.

    ReplyDelete
  4. Bob, according to DCPS this was a different test:
    At a standing room only meeting, Chancellor Henderson released DC CAS scores from the 2012-2013 academic school year. The DC CAS are a series of standardized tests aligned to the Common Core State Standards, and measure a student’s mastery of reading, mathematics, science, and writing composition. This year’s scores revealed that DCPS improved its proficiency in all subjects more rapidly than in any year since 2009. In just the second year of Chancellor Henderson’s ambitious five-year strategic plan, student outcomes across the district are improving as dramatically as planned. Although much work lies ahead, the data is compelling evidence that DCPS’s reforms are working.
    http://goo.gl/DHRxWp
    That wasn't the case last year.
    It's like the time from a decade ago when DCPS switched from Stanford 9 to DC-CAS.
    SY11-12 DC-CAS is not the same as SY12-13 DC-CAS.

    ReplyDelete