Part 3—A somewhat arcane report: Based upon our most reliable data, it's easy to see that Chicago's schools have been performing much better.
We refer to the National Assessment of Educational Progress (Naep), the long-standing federal testing program which is often called "America's Report Card."
There have been rampant cheating episodes on some state testing programs. Some state testing programs have been poorly constructed.
In the forty-plus years since the Naep's inception, it has been assumed that the Naep is competently constructed—and that it's much less susceptible to cheating or technical manipulation than the various state-run tests.
The Naep is commonly called "the gold standard" of educational testing. And it's easy to see, from Naep results, that the much-maligned Chicago Public Schools seem to have been headed in the right direction—in the last decade, let's say.
Unless something's wrong with the basic Naep data, Chicago's eighth graders have been making great strides in math. Here are Chicago's most recent average scores from the Grade 8 Naep math test:
Average scores, Grade 8 math, NaepBy 2017, the average eighth-grader in Chicago was scoring 17.5 points higher than her counterpart from twelve years before. Applying a standard, very rough rule of thumb, that represents a gain of something like 1.75 years in academic achievement.
All students, Chicago Public Schools
That would be a massive improvement. Meanwhile, this is what the numbers look like for Chicago's Hispanic kids, the system's largest demographic:
Average scores, Grade 8 math, NaepBy that very rough rule of thumb, Chicago's Hispanic eighth graders outperformed their counterparts from 2005 by roughly 1.35 academic years.
Hispanic students, Chicago Public Schools
We'll offer depressing context tomorrow, but those would be very large gains. It's also true that these score gains are much larger than the corresponding gains recorded by the nation's kids as a whole.
Where Hispanic kids in Chicago gained 13.5 points in math, Hispanic kids nationwide gained only six points over that twelve-year span. If we accept the basic worthiness of Naep data, Chicago's kids have been making enormous strides in math. They've been substantially outgaining their peers in the nation as a whole.
Along the way, Chicago has risen toward the top of the charts on the Naep among the fifteen or so comparable city systems which participate in the program's "Trial Urban District Assessment." Tomorrow, we'll offer examples of what we mean.
How has Chicago done this? Before we leave this toddlin' town, we'll show you what the New York Times' Emily Badger saw when she visited Lavizzo Elementary School. The visit was part of her reporting for the lengthy news report which appeared last December.
Badger described activity in that school which would make a decent person's heart sing. You haven't heard about this on your favorite corporate cable channel because, within that corporate realm, the massively overpaid corporate denizens don't care about this sort of thing, or about the beautiful kids who attend Chicago's schools.
If we assume the basic worth of Naep data, it's easy to see that Chicago's schools have had something going on. If our tribal heroes cared about black and Hispanic kids, they'd want to let us know about this. They'd very much want to know what is producing these changes.
When that lengthy news report by Badger and Kevin Quealy appeared, it of course generated exactly zero discussion. That said, it appeared beneath a hard-copy headline which was perhaps misleading:
New Measure Shows Where Students Learn the MostIn truth, Chicago really isn't the place "where students learn the most." We'll offer a ton of depressing context tomorrow.
That said, Chicago really may be one of the places where test scores have risen the most, in recent years, for individual student cohorts—for groups which proceed from the end of third grade through the end of eighth grade, five years later. Headline to the side, that was the specific claim in the Stanford study which was discussed in that New York Times report.
That claim can't be checked through data from the Naep, which is administered in Grades 4 and 8, and which generates data for only a few dozen individual school districts. (The Naep generates data for every state and for the nation as a whole, and for a few dozen large school districts.)
That said, we've compared Chicago's Grade 4 Naep scores in recent years to its Grade 8 Naep scores four years later, and we've found the same phenomenon reported by Stanford's Sean Reardon. The score gains recorded from Grade 4 to Grade 8 (four years later) are much larger than the corresponding gains recorded by the nation as a whole.
Reardon's metric in this recent study is perhaps a bit arcane or eccentric. Like almost everything else on earth, it has some obvious shortcomings, at least as it was reported in the Times piece.
In principle, the reliance on "aggregate" scores can hide a lot of sins. (We'll note an example tomorrow.) Meanwhile, the desire to work with Grade 3 scores requires use of the state testing programs on which massive cheating has sometimes occurred.
It's also true that no student cohort goes through five years intact. Some students will leave a big urban system during the five years after Grade 3. Other students will move into the system. The group you're testing in Grade 8 is not the same group you tested in Grade 3, five long years before.
Reardon's metric may not be the best for popular journalistic purposes. It's easy to show that Chicago's schools seem to be making major strides in the more straightforward, old-fashioned way, through comparisons llke this:
Average scores, Grade 8 math, NaepThose darn white kids have been rocking the world! With a bit of statistical jumping around, we're looking at very large score gains.
White students, Chicago Public Schools
If we accept Naep data as basically valid, the city of broad shoulders seems to be taking long strides. That said, is Chicago really the place "where students learn the most?"
You may already have noticed at least one problem. Tomorrow, we'll bring the note of sadness in, much as the poet described.
As we do, we'll point to the problem we all live with today. We'll point to a major challenge.
Tomorrow: Not even close to the most
You'll find few journalists there: For all Naep data, just click this. From there, you're on your own.