A commenter asks a good question: Almost surely, you will never see a serious discussion of our public school test scores.
The truth is, no one really cares about the truth of that matter. On the other hand, a lot of people in the press corps care about billionaires.
In this country, many well-known billionaires are committed to the standard gloomy stories in which our public schools just keep getting worse. Our schools were so much better back then! As James Atlas just said!
Almost surely, you'll never see a serious discussion of this topic. But just in case a young revolutionary is lurking somewhere, we want to record appropriate points about the way test scores should be discussed. This brings us to an excellent move Kevin Drum recently made.
And to a very good question.
Drum made his excellent move July 2, while showering us with the kind of praise we always find so embarrassing. His self-referential headline said this:
“Kevin Drum Smackdown Watch: I Forgot About Disaggregation!”
In his post, Drum explains why you have to “disaggregate” test scores if you want “to really understand what's going on.”
We agree with that overall judgment. For most purposes, you have to look at how different parts of the student population are doing. You can’t just look at the “aggregate” score. You have to see how Hispanic students did. You have to take a separate look at the scores of black students if you want to get a full picture of what is going on.
In his post, Drum explains why that’s so important. We were glad to see his post, because Drum has always tended to present aggregate scores when he reviews the NAEP.
Drum is one of the only liberals who seems to care about such topics. The analysts were glad to see him singing the praises of disaggregation.
Having said that, let us also say this. One of Drum's commenters posed a very good question that day. First, he quoted something Drum said, then he let her rip:
COMMENTER (7/2/13): “The rising share of blacks and Hispanics has pushed down the average when you lump everyone together.”In fact, overall test scores are improving, even before disaggregation. But the commenter asked a very good question. Here’s the answer:
OK, but why is it assumed to be obviously improper to "lump everyone together?" I should think it would depend on whether the question is, “Are scores failing to improve,” or the question is, “Why scores are failing to improve.” This explanation only goes to why, it doesn't undo the fact that scores are failing to improve.
It isn’t “obviously improper” to lump everyone together. Depending on the question you’re trying to answer, that may be the right thing to do.
But most of the time when we talk about test scores, we’re trying to evaluate the work of our schools, or the work of some particular school, or the value of some instructional program. In all such discussions, you really need to disaggregate scores, so that you end up comparing roughly similar groups of students.
If School A is full of kids from high-income professional homes, and School B is full of the children of low-income recent immigrants from low-literacy backgrounds, it doesn’t make sense, for most purposes, to compare their test scores. Given the way the world works today, School A will have much higher scores.
That doesn’t necessarily mean that School A is doing a better job. If the student populations are hugely different, the difference in test scores doesn't mean much of anything at all.
Generally, we talk about test scores in the search for new ways to beat up on teachers. If you’re going to use test scores to examine the work of our teachers, you really do need to “disaggregate” scores.
Aggregate scores are not unimportant, but they only tell part of a much larger story. When you break scores down in various ways, you give yourself a much richer idea of all that is going on.
You also skip past the obvious blunders the billionaires like to make.
Tomorrow: The final, shocking installment in our NAEP-watch reports