OUTSCORING FINLAND: On the TIMSS!

MONDAY, NOVEMBER 25, 2013

Part 1—When the gong show started: It all began in December 2001.

Early that month, the OECD released the first test scores from the Program for International Student Assessment (PISA), the brainchild it had devised to measure “critical thinking.”

On that somewhat unconventional measure, Finland scored at the top of the world, producing mild consternation. From that day to this, journalists from various countries have taken the free trip to Finland to attempt to figure out how the Finns do it.

(Or perhaps just to pretend to try to figure it out.)

Upon their return to their native lands, they type the standard conventional memorized scripts, generally displaying near-total cluelessness about the culture of literacy in the process. They marvel at Finland’s miraculous scores, then repeat a set of conventional claims about the way the miraculous Finns have achieved their miraculous levels of achievement.

As is often the case in these matters, a familiar pattern sometimes seems to prevail. If everyone is going to say all the same things, the only way a scribe can stand out if by overstating the standard narrative in some dramatic fashion.

(Example: In the fall of 1999, every pundit was warning the world about Al Gore’s troubling three-button suits, which were extremely disturbing. So Arianna Huffington went on TV and shrieked about his non-existent four-button suits.)

Back to miraculous Finland:

When everyone agrees to say the same things for twelve solid years, there may be a tendency to start overstating a tad. That may explain the way Amanda Ripley misstated the facts at the start of her ballyhooed book, The Smartest Kids in the World.

In the following passage, Ripley pretends to explain why she wrote her widely-praised book, for which she was funded by ruling elites. In the process, Ripley makes the biggest misstatement in the history of books:

RIPLEY (page 2): Then one day I saw this chart and it blew my mind.

The United States might have remained basically flat over time, but that was the exception, it turned out. Look at Finland! It had rocketed from the bottom of the world to the top, without pausing for breath. And what was going on in Norway, right next door, which seemed to be slip sliding into the abyss, despite having virtually no child poverty? And there was Canada, coming up from mediocrity to the heights of Japan. If education was a function of culture, could culture change that dramatically—that fast?

...Compared to most countries, the United States was typical, not much better nor much worse. But in a small number of countries, really just a handful of eclectic nations, something incredible was happening.

In that passage, Ripley is describing a chart which is shown on page 3 of her book. In the process, she makes the biggest misstatement in the history of books.

Did that chart really blow Ripley’s mind when she saw it for the first time? Beyond that, has something incredible happened in Finland over the fifty-year period portrayed on that chart?

In each case, we would be inclined to say no, though we don’t mean that as a criticism of Finland, a small, unicultural, middle-class nation which probably has a lot of very good schools.

In our judgment, nothing incredible has happened in Finland, although the country tends to score well on international tests. But that’s a matter of judgment. This isn’t:

Manifestly, Finland didn’t “rocket from the bottom of the world to the top, without pausing for breath,” on the international tests recorded on Ripley’s mind-blowing chart. (Are Ripley’s funders funding her acid?) On Ripley’s chart, Finland was already scoring higher, in the mid-1960s, than most of the nations portrayed on the the chart are scoring even today.

Finland was always a high-scoring nation, according to Ripley’s comically misdescribed chart. On page 2 of her book, Ripley came up with a ludicrous claim, although, in fairness, her ludicrous claim did make the miracle greater.

Go ahead! Try to find a single reviewer who cited this ludicrous error! In our post-journalistic culture, career “reporters” agree to extend the Official Approved Standard Tales.

Back to that more recent history:

In December 2001, Finland was unveiled as the top-scoring nation on the inaugural PISA. The junkets to Helsinki began—and the Finns stopped taking part in the Trends in International Mathematics and Science Study (TIMSS), a more conventional set of international tests which measure mastery of math and science curriculum rather than “critical thinking.”

In 2011, Finland finally returned to the TIMSS, which is given to students in Grades 4 and 8. This is the way the United States and Finland scored in math that year:

Average scores, 2011 TIMSS, math
Grade 4:
Finland 545
United States 541

Grade 8:
Finland 514
United States 509

On the TIMSS scale, 500 is set as the international average, with a standard deviation of 100. Finland did outscore the United States, but at each grade, the two countries’ scores were regarded as statistically indistinguishable.

In science, Finland outscored the U.S. by somewhat wider amounts, though no miracles are apparent (see below). In the next few days, we’re going to look at these scores in more detail, through several available lenses.

On the one hand, we’re going to look at the way nine individual states scored at the Grade 8 level. In 2011, nine individual states took the TIMSS at Grade 8 as if they were independent entities. As such, the scores achieved by those nine states can be directly compared to Finland’s.

We’re also going to look at these scores through the lens of so-called race.

As noted, Finland is a largely unicultural nation; the United States is not. In the course of our brutal American history, our benighted ancestors spent several centuries trying to eliminate literacy in one segment of the American population. The backwash of this brutal history still affects American schools.

It’s also true that the United States has a fairly large immigrant population. Finland has a very small number of (delightful, deserving) immigrant kids in its schools.

The TIMSS, like the PISA, is a major international test battery. What can we learn when we look at scores from Finland and the United States through these particular lenses?

Tomorrow, we’re going to look at the nine different states which took the Grade 8 TIMSS as independent entities. We’re going to see something the Ripleys never report or discuss, apparently because they’re in thrall to standard preferred elite narratives:

Ballyhooed miracles to the side, we’re going to see a bunch of states outscoring Finland. We’re going to ask when the people who pose as “reporters” plan to report educational miracles of this type.

Tomorrow: Nine states tackle Finland

TIMSS scores in science: The TIMSS is test of science and math. These are the scores from 2011 in science:

Average scores, 2011 TIMSS, science
Grade 4:
Finland 570
United States 544

Grade 8:
Finland 552
United States 525

In each case, the difference between the scores is regarded as statistically significant. That said, the differences are hardly the stuff of miracles.

Tomorrow, we’ll see a couple of states outscoring miraculous Finland in science. We’ll see six states (out of nine) outscoring Finland in math.