How not to report on the public schools!

THURSDAY, APRIL 19, 2012

An underwhelming report in the Post: Good grief—and heavens to Betsy!

In yesterday’s Washington Post, the world got a primer in how not to report on the public schools. Just on a simple technical basis, the work was depressingly bad.

The report concerned two small Washington-area school districts. Jeremy Borden—and his editors—thought they had spotted a problem, based on these districts’ test scores.

The report dominated the front page of the Post’s Metro section. Here’s how it began:
BORDEN (4/18/12): Manassas, Manassas Park schools have much in common, get different results

On the surface, the public schools in two small Northern Virginia cities share much in common.

Manassas and Manassas Park draw many students who speak limited English or come from poor families. Both are intimate school systems, with fewer than 10,000 students, overshadowed by the second-largest school system in the state in surrounding Prince William County.

Despite the similarities, the city known as “the Park” holds an edge over Manassas in student achievement.
Wow! These school systems seem to be a great deal alike—but one is getting better results! The caption to this report’s large photo called this “an unexpected disparity.”

That sounded like it could be a story. Then Borden presented some data:
BORDEN: About 85 percent of Manassas Park students pass state English tests, and 84 percent pass in math. The Park’s on-time high school graduation rate is 85 percent. In Manassas, the test pass rates are 78 percent for English and 79 percent for math, and the graduation rate is 77 percent.
Good grief! Those passing rates don’t differ by much! When Borden added a bit more data, our faces really fell:
BORDEN: Educators say there is no simple answer to the question of what separates Manassas Park, the smaller of the two, from Manassas.

Demographics could be a factor. About 45 percent of Manassas Park’s 3,019 students come from families poor enough to qualify for meal subsidies, but the rate is 54 percent among the 7,154 students in Manassas. Manassas Park has a high share of students with limited English proficiency—36 percent. But the rate in Manassas is even higher—41 percent.
Oof. Manassas is slightly poorer, with slightly more English learners—and its passing rates are slightly worse. That’s pretty much what you would expect, given the way low income and second-language issues routinely interact with achievement in American schools.

There are a few other factors in Borden’s report which might be worth exploring. (One example: “Manassas Park gets better results even with Manassas spending about $1,590 more per student.”) But this is underwhelming work on the part of the Post.

Tomorrow, we’re going to tell you what we saw when we went to kindergarten this week. Preview:

We were impressed with the glimmers we got. Not so much with this piece in the Post.

2 comments:

  1. One thing I noticed about that article is that it didn't mention books or other learning materials. Whether a student enjoys and can comprehend their school books is more important than where the mirrors are. Big textbook publishing companies have way too much influence in today's schools, and teachers would be happier and more effective if they had a larger role in choosing the materials.

    ReplyDelete
  2. What is the confidence interval in this comparison? Is the author confident that the difference is a real difference or the results are within the statistical margin of error? Why do they write such stuff without analysis? (spouting facts is not analysis, unfortunately).

    ReplyDelete