EXTRAORDINARY RESULTS: How extraordinary were those results?

THURSDAY, APRIL 25, 2019

Lack of clarity all the way down: If we might adapt a mocking rubric, Erica Green's front-page report was driven by claims which were apparently "close enough for New York Times public school work."

Green was discussing Akron's I Promise School, an experimental new elementary school which is being substantially funded by NBA star LeBron James. As is common when the Times pretends to discuss public schools, Green's extraordinarily deceptive claims came thick and fast.

She opened with a flat misstatement, claiming that the new school's students "were identified as the worst performers in the Akron public schools" before being assigned to I Promise.

Based on something Green reports in paragraph 23, it seems that claim is flatly false. Based on what Green wrote, we'll guess that the lowest-performing twenty percent of Akron's public school kids were excluded from the lottery which sent kids to the I Promise School.

Green opened with that flat misstatement about the I Promise kids. She closed her report with this astonishing feel-good tale:

GREEN (4/13/19): Lining the walls of the school’s vast lobby are 114 shoes, including those worn during the 2016 season when Mr. James led the Cleveland Cavaliers to the N.B.A. championship, a reminder that he once walked a path similar to these students. Mr. James was also considered at risk; in fourth grade, he missed 83 days of school.

Nataylia Henry, a fourth grader, missed more than 50 days of school last year because she said she would rather sleep than face bullies at school. This year, her overall attendance rate is 80 percent.

“LeBron made this school,” she said. “It’s an important school. It means that you can always depend on someone.”

Astonishing! If that fourth grader is missing school 20 percent of the time this year, that doubles the rate the state of Ohio defines as "chronic absenteeism!" Unless you're reading the New York Times, where a (deliberately?) jumbled presentation makes it sound like her attendance rate is cause for jubilation.

What can anyone say about a newspaper which published work of this type? We'd say the paper is "Trump-before-Trump"—that Green's report is an example of the rolling moral and intellectual disorder now widely described as "Trumpism."

The Times has behaved this way forever, though very few liberals know this. Over Here in our self-impressed tribe, we're told that the dummies are all in the other tribe, and that the New York Times is our nation's smartest newspaper.

Because we're deeply tribal animals, we're unable to see that that's wrong.

Green opened with what seems to be a flatly false claim, closed with a feel-good gong-show. Along the way—early on—she made this extraordinary set of claims:

GREEN: This time last year, the students at the school—Mr. James’s biggest foray into educational philanthropy—were identified as the worst performers in the Akron public schools and branded with behavioral problems. Some as young as 8 were considered at risk of not graduating.

Now, they are helping close the achievement gap in Akron.

The academic results are early, and at 240, the sample size of students is small, but the inaugural classes of third and fourth graders at I Promise posted extraordinary results in their first set of district assessments. Ninety percent met or exceeded individual growth goals in reading and math, outpacing their peers across the district.

These kids—supposedly, they were Akron's "worst performers" last year—have now "posted extraordinary results in their first set of district assessments."

They're "helping close the achievement gap in Akron," Times subscribers were told. This is a familiar old tale, but is any part of it true?

It seems quite clear that these kids didn't start out as their city's "worst performers," or as anything close to that status. It seems quite clear that at least one of the new school's fourth graders is missing a lot of school, though the Times chose to reinvent that situation as an upbeat, feel-good fable.

That said, is Green's overall presentation accurate? Have the I Promise kids "posted extraordinary results" in some set of "district assessments?"

This pleasing claim drives Green's front-page report throughout. But is it actually true?

A savvy reader will note one point—in that first presentation, Green refers to how well these kids allegedly did with reference to meeting some undefined set of "growth goals." She doesn't tell us how well they did in meeting some set of objective academic standards.

That presentation in paragraphs 3-5 is rather murky. Later, in paragraphs 12-14, Green defines, or attempts to define, the nature of the "extraordinary results" which form the heart of her report:

GREEN: The students’ scores reflect their performance on the Measures of Academic Progress assessment, a nationally recognized test administered by NWEA, an evaluation association. In reading, where both classes had scored in the lowest, or first, percentile, third graders moved to the ninth percentile, and fourth graders to the 16th. In math, third graders jumped from the lowest percentile to the 18th, while fourth graders moved from the second percentile to the 30th.

The 90 percent of I Promise students who met their goals exceeded the 70 percent of students districtwide, and scored in the 99th growth percentile of the evaluation association’s school norms, which the district said showed that students’ test scores increased at a higher rate than 99 out of 100 schools nationally.

The students have a long way to go to even join the middle of the pack. And time will tell whether the gains are sustainable and how they stack up against rigorous state standardized tests at the end of the year. To some extent, the excitement surrounding the students’ progress illustrates a somber reality in urban education, where big hopes hinge on small victories.

In that passage, we learn that these "extraordinary results" really involve "small victories." So it goes when the New York Times pretends to talk about schools.

To call that overall presentation "fuzzy" is to be unfair to fuzz. Meanwhile, Green notes that the I Promise kids haven't even taken their first set of annual Ohio statewide tests.

The notion that such statewide tests are "rigorous" may be Green's latest stretch. But why on earth would a major newspaper present a front-page evaluation of a brand-new school before its kids had taken their first set of annual statewide tests?

Presumably, the Times did that because it likes a certain type of feel-good story about miracle cures in low-income schools—and because a famous celebrity was part of the mix.

Otherwise, this whole undertaking makes almost no sense. That said, let's try to see how "extraordinary" those results actually were:

Achievement by percentile:

Green reports that she's describing results from the Measures of Academic Progress assessment, a relatively obscure new test battery. She says the kids at I Promise School have achieved these results:

Grade 3 reading: 9th percentile
Grade 3 math: 18th percentile
Grade 4 reading: 16th percentile
Grade 4 math: 30th percentile

Might we talk? To the naked eye, those don't exactly look like "extraordinary results."

Though Green never quite explains, these data seems to mean that 91 percent of schools nationwide outscored the I Promise third-graders in math. If we average the I Promise School's performance in those four measures, we see that the average result places I Promise kids in the 17th percentile nationwide.

On average, 83 percent of the nation's schools outperformed the I Promise kids! Does a familiar phrase come to mind? "The soft bigotry of low expectations?"

Claims of improvement:

Truth to tell, those achievement levels don't look all that great. But hold on! The point, says Green, is the massive amount of improvement the I Promise kids have recorded!

Once again, here's the relevant passage:

GREEN: The students’ scores reflect their performance on the Measures of Academic Progress assessment, a nationally recognized test administered by NWEA, an evaluation association. In reading, where both classes had scored in the lowest, or first, percentile, third graders moved to the ninth percentile, and fourth graders to the 16th. In math, third graders jumped from the lowest percentile to the 18th, while fourth graders moved from the second percentile to the 30th.

"In math, third graders jumped from the lowest percentile to the 18th." That certainly sounds like an improvement—but as with almost everything in this report, we're not entirely sure what that means.

Does that mean that the I Promise third-graders scored in the first percentile when they were in second grade? That's what we initially thought this passage meant—but since the I Promise kids were in various schools last year, the claim wouldn't fit any standard type of before-and-after assessment.

Today, it occurs to us that Green might mean something different. She may mean that the I Promise third-graders took a Measures of Academic Progress "pretest" at the start of this school year, then moved to a higher percentile on a recent re-testing.

As with almost everything in this report, it's hard to know what Green is talking about in this passage. But just for the record, there's a possible problem with that pretest/re-test model.

As we'll note below, this possible problem which didn't bark involves a naughty vocabulary word—a word which starts with a "C."

Claims of attainment of "goals:"

As Green continues, she makes an even murkier statement about the "extraordinary results" produced by the I Promise kids. This involves the number of kids who "met their goals:"

GREEN: The 90 percent of I Promise students who met their goals exceeded the 70 percent of students districtwide, and scored in the 99th growth percentile of the evaluation association’s school norms, which the district said showed that students’ test scores increased at a higher rate than 99 out of 100 schools nationally.

According to Green, 90 percent of the I Promise kids "met their goals," apparently on the poorly-described Measures of Academic Progress assessment.

She doesn't explain who set these goals, or how rigorous these goals might have been.

Anyone can meet a goal if the goal is easy enough! According to Green, a certain near-paradox obtains:

Something like 90% of the I Promise third graders managed to land (as a group) in the 9th percentile nationwide as a group, even while meeting their individual goals!

That said, this too is claimed: While 90% of the I Promise kids met their goals, only 70% of kids met their goals in Akron as a whole.

For better or worse, doing somewhat better than Akron kids as a whole may not be a massive achievement. On its largely indecipherable "Ohio School Report Card," the Akron Public Schools currently receive a D for Achievement and an F for Progress.

That's based on scores from last year's "rigorous" statewide tests, which haven't been given yet this year.

As noted, this lets everyone blather about how "extraordinary" these early I Promise results have been. A cynic would say that the Times may be blathering while the blathering's good.

We mention a terrible word:

It's very, very hard to know what kinds of "extraordinary results" Green is discussing in this report. It's amazing to think that a major newspaper would go "front page" with piddle like this before a new school's first year is completed—before the amazing new school has administered its first set of statewide tests.

That said, Green began her report with an outright falsehood, then ended it with a scam. Along the way, she also quoted a range of school and foundation officials lavishing praise on themselves, a standard component of miracle press corps reports of this heart-warming type.

A range of officials praise their own greatness. Along the way, one district official is quoted making a slightly awkward remark:

GREEN: “It’s encouraging to see growth, but by no means are we out of the woods,” said Keith Liechty, a coordinator in the Akron public school system’s Office of School Improvement. The school district, where achievement and graduation rates have received failing marks on state report cards, has been trying to turn around its worst-performing schools for years. “The goal is for these students to be at grade level, and we’re not there yet. This just tells us we’re going in the right direction,” he added.

But Mr. Liechty, who has been with the district for 20 years, said that the students’ leaps would not be expected in an entire school year, let alone half of one. “For the average student,” he said, “your percentile doesn’t move that much unless something extraordinary is happening.”

To Leichty's credit, he tries to put the brakes on the premature celebrations. But then, he makes an awkward remark:

He says you wouldn't expect this much improvement in an entire year, let alone in half a year. “For the average student,” he says, “your percentile doesn’t move that much unless something extraordinary is happening.”

For the record, Green hasn't cited percentiles for any individual students. She only seems to be citing percentiles for the I Promise School as a whole.

That said, Leichty makes a slightly scary remark. Across the country, again and again, when "extraordinary results" have appeared, the apparently extraordinary improvement has involved the C-word:

Cheating.

Is anyone cheating at this new school? We have no idea. We have no reason to think so.

That said, if these results are built on a pretest/re-test model, were both sets of tests administered in an appropriate way?

We can't answer that question either. We can tell you this:

When "extraordinary results" occur, journalists should trust but verify. In other words, they should be suspicious.

They should remember the giant cheating scandals in Atlanta and Washington. But expecting our "journalists" to behave that way is like expecting the overweight cow to leap over the moon.

Erica Green should have asked hard questions about this school's testing procedures. The re-test percentiles are still so low that any possible cheating would apparently have been minimal. But the "pre-test" percentiles were rock-bottom.

A real reporter might have wondered where those rock-bottom initial scores came from. If a misadministered pretest produced rock bottom scores, that might explain why the I Promise test scores "increased at a higher rate than 99 out of 100 schools nationally."

The New York Times, a Trumpian guild, will never behave in such ways. If Trump is dumb, the Times is dumber. This famous newspaper has been behaving this way for a very long time.

In this case, Green's report began with a blatant false claim and ended with an act of journalistic fraud. Along the way, her various claims were poorly explained and very hard to follow.

Our liberal world just sat and smiled. We've been this way for years.

Tomorrow: Columbus, we have a problem!