The Washington Post gets its pom-poms out!

SATURDAY, NOVEMBER 9, 2013

Editorial board cheerleads/misleads about those test score gains: The Washington, D.C. public schools recorded good score gains on the 2013 NAEP.

In yesterday’s featured editorial, the paper’s editorial board got its pom-poms out. They did some of their familiar cheerleading for “education reform.”

They also offered a misleading picture of score gains in D.C.

This is the way the editorial began, headline included. Much of this is technically accurate. Much of this is misleading
WASHINGTON POST EDITORIAL (11/8/13): High marks for D.C. schools

School reform in the District is working. That is the unassailable message of test scores released Thursday by federal education officials. Students at every level improved in reading and math, and the improvement exceeded the national average. Tremendous gains in the past two years show that there has been no lessening in the intensity of school reform under Mayor Vincent C. Gray (D) and Schools Chancellor Kaya Henderson.

"A pretty remarkable story" was the apt summary by Education Secretary Arne Duncan, who joined D.C. officials in celebrating the results of the 2013 National Assessment of Education Progress (NAEP). This respected report card showed gains for District students of double digits in math and of nearly 10 points in reading since 2007, when the reforms began. Striking gains were achieved since 2011: five points in fourth-grade reading; seven points in fourth-grade math; six points in eighth-grade reading; and five points in eighth-grade math.

The NAEP results include both the traditional school system and public charter schools. While a significant achievement gap persists between minority students and white students, all subgroups of students, save English-language learners, improved. And while D.C. students still trail the national average, the steady, fast improvement puts them on pace to catch up.

In other words: The reforms started during the administration of former mayor Adrian M. Fenty (D) and schools chancellor Michelle A. Rhee, and sustained and refined under Mr. Gray and Ms. Henderson, are working. They are based on high standards, rigorous evaluation of teachers, an investment in pre-kindergarten and school choice.
As noted, the NAEP results for D.C. include traditional public schools and the city’s many public school charters. Test scores rose, by healthy amounts, in the past two years.

Does that mean what the editors say? Does that mean “reform is working?” The editors oversimplify here, as they constantly do when they take their pom-poms out.

D.C. has a very unusual student population. White kids make up about ten percent of the student population. As a group, they tend to come from affluent, highly literate backgrounds.

They are the children of think tanks. For that reason, D.C.’s white kids outscore their counterparts in each of the fifty states. They’re the highest-scoring student cohort in the entire nation.

That said, the heart of the D.C. system, the system’s reason for being, is its large population of black kids. Their test scores were already rising before the current reforms took effect.

That editorial tracks score gains back to 2007, “when the reforms began.” This truncated history lets the editors get their pom-poms out.

The gains since 2007 have been pretty good. But below, you see the average scores in Grade 8 math for D.C.’s black kids dating to 2000, the first year in which scores are available.

All scores in today's post come from the NAEP. According to a very rough rule of thumb, ten points on the NAEP scale is often compared to one academic year:
Black students, Washington D.C.
Grade 8 math, NAEP

2000: 231
2003: 240
2005: 241
2007: 245
2009: 249
2011: 256
2013: 261
The average score has risen by 16 points in the six years since 2007. But the average score rose by 14 points in the seven years before that.

If we apply that very rough rule of thumb, those are large score gains—and the pattern predates 2007. A similar pattern obtains in Grade 4 math:
Black students, Washington D.C.
Grade 4 math, NAEP

2000: 188
2003: 202
2005: 207
2007: 209
2009: 213
2011: 215
2013: 221
That first score looks suspiciously low. (Sampling errors do happen.) But if we go to war with the test scores we have, score gains were actually larger before 2007.

We don’t say this to knock the efforts which have been made since 2007. A lot of people have worked hard in the D.C. schools. They've had good ideas and bad.

We don’t say this to criticize them. We say this to criticize the Post’s editorial board, who have always behaved like cheerleaders in this arena, misserving the public interest.

Just to complete the picture, here are the reading scores for black kids during those years. As has been true all over the nation, the score gains have been smaller in reading than in math. But again, the score gains were already happening in the years before reform:
Black students, Washington D.C.
Grade 4 reading, NAEP

1998: 174
2002: 188
2003: 184
2005: 187
2007: 192
2009: 196
2011: 193
2013: 197

Black students, Washington D.C.
Grade 8 reading, NAEP

1998: 233
2002: 238
2003: 236
2005: 235
2007: 238
2009: 239
2011: 239
2013: 243
Again, that 1998 Grade 4 score looks suspiciously low. From 1998 through 2007, the Grade 4 score bumps around in a peculiar pattern.

But score gains were being recorded before the watershed year of 2007. The board should stop playing cheerleader in this arena and start behaving like journalists.

Final note: D.C. test scores are relatively hard to interpret for several reasons. There is a churn between traditional schools and public charters which can create some confusion. Changes in policy concerning free lunch have made some comparisons tricky or impossible. The return of some high-scoring white students to the schools, traditional and charter, can obscure the overall picture.

(Emma Brown, in Friday’s news report: “White students now account for 10 percent of fourth-graders, up from 8 percent just two years ago, while the proportion of black fourth-graders fell from 77 to 73 percent during the same period.”)

D.C. test scores can be tricky. But according to Brown, black kids comprised 73 percent of the fourth-grade student testing this year. They are the heart of the D.C. schools—and they recorded good score gains on this year’s tests.

That said, those score gains are part of a pattern which predates 2007. The board should put its pom-poms away and pledge to present “the whole truth.”

15 comments:

  1. Oh, my God, the media are all partisan bots!

    What's happening that we can keep going?

    ReplyDelete
    Replies
    1. Insane trolling, but you know that.

      Delete
    2. This would be a good time to call for blocking the trolls, but there seem to be none here. Nobody else either. Guess only those who know something about pom poms came.

      Delete
  2. Important and much needed criticism, Bob.

    ReplyDelete
  3. It does seem to me that more improvement happened after 2007 for black children. The scores bounced around a lot, but they bounced higher after 2007.

    ReplyDelete
  4. This is a case for statistical analysis. Specifically, two hypothesis testing or linear regression can determine if the changes since 2007 are due to eduction reform, or to mere chance. This type of testing is done all the time, for example when testing a new drug. I have to do such testing as part of my work.

    I don't have the software on my home computer, and it takes a bit of time to set up, but I am pretty certain that there is no statistical significant change with the data before 2007 and after 2007.

    (We have very small data sets here, so in order for the second data set to statistically differ from the first, the second would have to be dramatically different. Eye balling that, I don't see that at all.)

    ReplyDelete
  5. This is a case for statistical analysis. Specifically, two hypothesis testing or linear regression can determine if the changes since 2007 are due to eduction reform, or to mere chance. This type of testing is done all the time, for example when testing a new drug. I have to do such testing as part of my work.

    I don't have the software on my home computer, and it takes a bit of time to set up, but I am pretty certain that there is no statistical significant change with the data before 2007 and after 2007.

    (We have very small data sets here, so in order for the second data set to statistically differ from the first, the second would have to be dramatically different. Eye balling that, I don't see that at all.)

    ReplyDelete
  6. Bob, wait until you see Maureen Dowd's latest. Now, she uses Sarah Silverman to complain about Hillary Clinton's voice! Talking about grasping at straws in order to whine again about Hillary!

    ReplyDelete
    Replies
    1. I'm sure Bob will spend an entire week on it.

      Delete
    2. I don't know if you are right Anon. @ 1:48. But I will conult my bald spot in the morning and give you odds, hopefully before the first post. Baldie thinks better in the morning.

      Delete
    3. BTW MR. Snipes, knowledgeable folks around here say Bob won't even see your question.

      Delete
    4. Anon@1:48

      Baldie says bet on one scathing post. He senses Somerby scents fresh meat in the educational/journalism field. Of course, being on the back of my head he may have peeked at the computer before we spoke.

      Delete
  7. Stop looking at the scores! Look at what's going on in the classroom because of high stakes testing! Simply put - it's not the quality education you would want for yourself or for your child!

    The narrow focus on testing is destroying public education.

    Ask a teacher.

    ReplyDelete
  8. The Washington Post has no credibility when it comes to education editorializing. None. Zilch.

    The Post’s education editorial write Jo-Ann Armao has written falsely that American public education is broken and teacher unions are to blame. Armao promotes charter schools and the abolishment of tenure as the “fix.”
     
    Armao subscribes to flimsy research on New York city charter schools to buttress her push for more charters in the District and elsewhere.  And if unions impede student learning, how does Armao explain high achievement in many strong union states like Maryland and Massachusetts and bottom-of-the-barrel achievement where unions are weak or non-existent (think, for example, Alabama and Mississippi)?

    Jo-Ann Armao showered praise on Michelle Rhee as DC schools chancellor, and scolded her critics, even when it increasingly appeared that the Rhee "miracle" was a mirage.

    A USA Today investigation into cheating in the District schools under Rhee found that for a school to be "flagged" for possible cheating a "classroom had to have so many wrong-to-right erasures that the average for each student was 4 standard deviations higher than the average for all D.C. students in that grade on that test, meaning that " a classroom corrected its answers so much more often than the rest of the district that it could have occurred roughly one in 30,000 times by chance. D.C. classrooms corrected answers much more often.”

    When half of all the schools in the system are flagged for grossly abnormal wrong-to-right erasures on tests and " the odds are better for winning the Powerball grand prize than having that many erasures by chance," then it's it’s almost certain that more than just "some cheating" took place.

    Yet, in a fit of obtuseness, The Post (Armao) opined that "there are many innocent explanations for changed answers." Worse, The Post told the public, inaccurately, that Rhee and her cronies "were cleared by an outside firm." None of it was true.

    Jo-Ann Armao has written a whole series of education editorials that are, to be kind, not worth the paper on which they are printed. Armao and The Post endorse charters, more testing, merit pay for teachers, and vouchers. There is little or no research to support any of them.

    What’s “remarkable” is that The Post (Armao) keeps grinding out the falsehoods rather than adhering to founder Eugene Meyer’s guiding principle that “The first mission of a newspaper is to tell the truth as nearly as the truth can be ascertained.”

    democracy

    ReplyDelete