WHERE THE PISA GAPS ARE: Substantial international gaps!

WEDNESDAY, OCTOBER 12, 2016

Part 1 in this series

Part 2—The PISA conflicts with the TIMSS: In December 2012, a quadrennial event occurred.

Results were released from one of the developed world's two major international public school testing programs. The testing program in question was the Trends in International Math and Science Study (the TIMSS), which includes a companion test in reading achievement, the PIRLS.

The TIMSS had released results from its most recent international testing, which had been conducted in 2011. Given the nature of the results, American newspapers faced a choice:

Should they describe the glass as 10 percent empty? Or should they describe it as being 90 percent full?

Needless to say, the New York Times took the gloomier, mandated route. Because USA Today's Greg Toppo presented a more inclusive summary of the results, we'll enter the start of his report into evidence.

He took the road less traveled by. Upbeat headline included:

TOPPO (12/11/12): USA's schools move up in international rankings

Results from a pair of new international assessments released today show that American kids are holding their own in math, reading and other subjects. In a few cases, they're actually bypassing the rest of the world.

Who knew, for instance, that Florida fourth-graders now read as well as their peers in Singapore and Finland?

[...]

American students' average scores on the Third International Mathematics and Science Study (TIMSS) in math were above the international average in both fourth and eighth grade, the findings show.

Among the 45 countries that participated in fourth grade, the average U.S. math score was among the top eight. In eighth grade, the USA was among the top 11 of 38 countries.

In reading, U.S. students scored 56 points higher than the international average, putting them in the top 13 on the Progress in International Reading Literacy Study (PIRLS). Only five nations or education systems had higher average scores—and one of those was Florida, which asked that its scores be compared with those of other nations.

With respect to the PIRLS, Toppo's reference to "the top 13" may have seemed to conflict which his more straightforward claim, in which "only five nations or education systems had higher average scores" than the United States. (In his claim about "the top 13," Toppo was including statistical ties.)

That said, you can see the full list of PIRLS scores here, with only Russia, Finland and Singapore (narrowly) outscoring the U.S. among the world's actual nations. Beyond that, you can derive a second point from Toppo's generally accurate, upbeat summary:

If we look at these, the most recent TIMSS/PIRLS scores, it's hard to support the familiar claim that U.S. schools, and the children and teachers within them, are a pitiful embarrassing mess.

American students outperformed their peers in most of the world on these international tests. Back in December 2012, Toppo gave an accurate summary of those new results.

Gloom and doom at the New York Times to the side, American kids seemed to have performed rather well on these TIMSS/PIRLS tests. They even came amazingly close to matching the scores of their peers in miraculous Finland, the small, middle-class European nation which had become the darling of propagandists and education journalists shortly after the start of this century.

If we had only the TIMSS and the PIRLS, advocates of certain types of education reform would be facing an uphill climb. In these most recent results from those tests, our American kids didn't seem to be a hopeless comparative mess.

Indeed, a sensible person might have found himself wondering about the educational performance of other nations. Why did American kids, in Grade 4 and Grade 8, create statistical ties with miraculous Finland in math? These were the results in question:

Average scores, Grade 4 math, 2011 TIMSS
Finland: 545
United States: 541

Average scores, Grade 8 math, 2011 TIMSS
Finland: 514
United States: 509

Given the way the TIMSS scale works, those differences are minor. From those scores, a sensible person might actually wonder what's the matter with Finland, a nation with much less poverty than the U.S., many fewer immigrant kids, and no national history in which centuries were spent trying to eliminate literacy from one segment of the population.

American kids had matched their peers from Finland in math? Given the demographic differences, a sensible person might have wanted to know why Finland hadn't scored better!

Why didn't Finland's kids score better on those TIMSS math tests? We can't tell you that. At least one Finnish math professor has complained about certain emphases in Finnish educational practice—emphases which, he has said, send Finnish kids into college with a shaky foundation in the basic math curriculum.

That claim to the side, we can't tell you how a major continental nation like the U.S. managed to match little middle-class Finland in math. We can tell you this:

Thanks to results from the world's other international testing program, such questions don't have to be asked by our education reporters.

That other international program is the Program for International Student Assessment (the PISA), which test 15-year-old students. When the PISA released its latest results, those from its 2012 testing, American kids stacked up less well as compared to the rest of the world.

When you hear the standard gloomy account of how poorly our hapless children perform on international tests, you're almost surely hearing about the PISA. Results from the TIMSS will go unmentioned, in line with prevailing standard elite press corps scripts.

How did American students perform as compared to the rest of the world on the 2012 PISA? Especially in math, their comparative performance wasn't especially good at all.

In reading and in science, the average score of American students basically matched the OECD average. In math, the average score of American students didn't match up even that well.

Because math is the subject in which U.S. kids performed least well on the PISA, that result is most frequently cited. Among OECD nations, here's how some of those math scores looked. We're including the larger OECD nations, plus a few of the usual suspects:

Average scores, Math literacy, 2012 PISA
Taiwan: 560
South Korea: 554
Japan: 536
Netherlands: 523
Finland: 519
Canada: 518
Germany: 514
Australia: 504
Denmark: 500
France: 495
United Kingdom: 494
OECD average: 494
Norway: 489
Italy: 485
Spain: 484
Russia: 482
United States: 481
Sweden: 478

That wasn't a great-looking performance. For full results, click this.

Remember, that was the worst performance by American students on the 2012 PISA. In "reading literacy" and "science literacy," American students basically matched the OECD average scores, and stacked up better as compared to their peers from other specific nations.

(For reading results, click here. For science results, click this.)

That was the worst performance by American students. Still, American students stacked up less well than they had on the TIMSS on all three parts of the PISA. Perhaps for that reason, the American public is constantly told about results from the PISA. Results from the TIMSS are routinely disappeared.

Does that selective presentation make some sort of sense? On the other hand, might it simply be a bow to certain current narrative preferences, in which certain elites prefer to advance a gloomy portrait of American schools?

We can't answer that question. You'll virtually never see it argued, so strong is the mandated journalistic preference for the gloom-and-doom perspective, in which the public is told about the PISA alone.

(In the realm of domestic testing, it's as we noted earlier in this series. The public is told about the achievement gaps, with the very large gains in scores being disappeared. The press corps' preference for gloom-and-doom is ruthlessly enforced. As in so many areas, the narrative must be advanced!)

American students seemed to score rather well on the TIMSS, substantially less well on the PISA. Perhaps because of the mandates of elite preference, the American public hears about the PISA results, with TIMSS results finding their way to the memory hole.

Does that selective presentation of data actually make some sort of sense? We'll briefly ponder that question tomorrow. But this is our week for reviewing the achievement gaps which exist in the PISA results.

On the most recent PISA, some large achievement gaps obtained between American kids and their peers in other nations. Tomorrow, we'll stay on that PISA road, and we'll take a horrible twist.

Tomorrow: Disaggregating the PISA!