OUTSCORING FINLAND: On the TIMSS!

MONDAY, NOVEMBER 25, 2013

Part 1—When the gong show started: It all began in December 2001.

Early that month, the OECD released the first test scores from the Program for International Student Assessment (PISA), the brainchild it had devised to measure “critical thinking.”

On that somewhat unconventional measure, Finland scored at the top of the world, producing mild consternation. From that day to this, journalists from various countries have taken the free trip to Finland to attempt to figure out how the Finns do it.

(Or perhaps just to pretend to try to figure it out.)

Upon their return to their native lands, they type the standard conventional memorized scripts, generally displaying near-total cluelessness about the culture of literacy in the process. They marvel at Finland’s miraculous scores, then repeat a set of conventional claims about the way the miraculous Finns have achieved their miraculous levels of achievement.

As is often the case in these matters, a familiar pattern sometimes seems to prevail. If everyone is going to say all the same things, the only way a scribe can stand out if by overstating the standard narrative in some dramatic fashion.

(Example: In the fall of 1999, every pundit was warning the world about Al Gore’s troubling three-button suits, which were extremely disturbing. So Arianna Huffington went on TV and shrieked about his non-existent four-button suits.)

Back to miraculous Finland:

When everyone agrees to say the same things for twelve solid years, there may be a tendency to start overstating a tad. That may explain the way Amanda Ripley misstated the facts at the start of her ballyhooed book, The Smartest Kids in the World.

In the following passage, Ripley pretends to explain why she wrote her widely-praised book, for which she was funded by ruling elites. In the process, Ripley makes the biggest misstatement in the history of books:
RIPLEY (page 2): Then one day I saw this chart and it blew my mind.

The United States might have remained basically flat over time, but that was the exception, it turned out. Look at Finland! It had rocketed from the bottom of the world to the top, without pausing for breath. And what was going on in Norway, right next door, which seemed to be slip sliding into the abyss, despite having virtually no child poverty? And there was Canada, coming up from mediocrity to the heights of Japan. If education was a function of culture, could culture change that dramatically—that fast?

...Compared to most countries, the United States was typical, not much better nor much worse. But in a small number of countries, really just a handful of eclectic nations, something incredible was happening.
In that passage, Ripley is describing a chart which is shown on page 3 of her book. In the process, she makes the biggest misstatement in the history of books.

Did that chart really blow Ripley’s mind when she saw it for the first time? Beyond that, has something incredible happened in Finland over the fifty-year period portrayed on that chart?

In each case, we would be inclined to say no, though we don’t mean that as a criticism of Finland, a small, unicultural, middle-class nation which probably has a lot of very good schools.

In our judgment, nothing incredible has happened in Finland, although the country tends to score well on international tests. But that’s a matter of judgment. This isn’t:

Manifestly, Finland didn’t “rocket from the bottom of the world to the top, without pausing for breath,” on the international tests recorded on Ripley’s mind-blowing chart. (Are Ripley’s funders funding her acid?) On Ripley’s chart, Finland was already scoring higher, in the mid-1960s, than most of the nations portrayed on the the chart are scoring even today.

Finland was always a high-scoring nation, according to Ripley’s comically misdescribed chart. On page 2 of her book, Ripley came up with a ludicrous claim, although, in fairness, her ludicrous claim did make the miracle greater.

Go ahead! Try to find a single reviewer who cited this ludicrous error! In our post-journalistic culture, career “reporters” agree to extend the Official Approved Standard Tales.

Back to that more recent history:

In December 2001, Finland was unveiled as the top-scoring nation on the inaugural PISA. The junkets to Helsinki began—and the Finns stopped taking part in the Trends in International Mathematics and Science Study (TIMSS), a more conventional set of international tests which measure mastery of math and science curriculum rather than “critical thinking.”

In 2011, Finland finally returned to the TIMSS, which is given to students in Grades 4 and 8. This is the way the United States and Finland scored in math that year:
Average scores, 2011 TIMSS, math
Grade 4:
Finland 545
United States 541

Grade 8:
Finland 514
United States 509
On the TIMSS scale, 500 is set as the international average, with a standard deviation of 100. Finland did outscore the United States, but at each grade, the two countries’ scores were regarded as statistically indistinguishable.

In science, Finland outscored the U.S. by somewhat wider amounts, though no miracles are apparent (see below). In the next few days, we’re going to look at these scores in more detail, through several available lenses.

On the one hand, we’re going to look at the way nine individual states scored at the Grade 8 level. In 2011, nine individual states took the TIMSS at Grade 8 as if they were independent entities. As such, the scores achieved by those nine states can be directly compared to Finland’s.

We’re also going to look at these scores through the lens of so-called race.

As noted, Finland is a largely unicultural nation; the United States is not. In the course of our brutal American history, our benighted ancestors spent several centuries trying to eliminate literacy in one segment of the American population. The backwash of this brutal history still affects American schools.

It’s also true that the United States has a fairly large immigrant population. Finland has a very small number of (delightful, deserving) immigrant kids in its schools.

The TIMSS, like the PISA, is a major international test battery. What can we learn when we look at scores from Finland and the United States through these particular lenses?

Tomorrow, we’re going to look at the nine different states which took the Grade 8 TIMSS as independent entities. We’re going to see something the Ripleys never report or discuss, apparently because they’re in thrall to standard preferred elite narratives:

Ballyhooed miracles to the side, we’re going to see a bunch of states outscoring Finland. We’re going to ask when the people who pose as “reporters” plan to report educational miracles of this type.

Tomorrow: Nine states tackle Finland

TIMSS scores in science: The TIMSS is test of science and math. These are the scores from 2011 in science:
Average scores, 2011 TIMSS, science
Grade 4:

Finland 570
United States 544

Grade 8:
Finland 552
United States 525
In each case, the difference between the scores is regarded as statistically significant. That said, the differences are hardly the stuff of miracles.

Tomorrow, we’ll see a couple of states outscoring miraculous Finland in science. We’ll see six states (out of nine) outscoring Finland in math.

12 comments:

  1. Always glad to learn the latest standardized test results from Finland.

    ReplyDelete
    Replies
    1. Aren't you also glad to know that US schools aren't as bad as they're always made out to be, especially compared to Finland?

      Teachers care about this because we are the ones whose unions are vilified, whose salaries are cut and who are laid off in the name of school reform. This matters to parents too, who wonder why their local schools are fine and yet people are always criticizing education. But you don't care, so no one should apparently.

      Delete
    2. I guess some people need Somerby to tell them that, but I already knew U.S. schools were good, particularly if you live in the lily-white suburbs or have the means to send your youngsters to private schools.

      But if you needed Somerby to say so before you'd believe it, then throw up a couple of numbers that really don't mean squat comparing the U.S. to Finland, well, as he used to say, continue lowing on the sweet hay he is throwing your way.

      Delete
    3. That is his point -- comparisons to Finland are specious. But Somerby isn't comparing the US to Finland -- Ripley is doing that, as are numerous other education journalists.

      Delete
    4. Anon @ 10:02 and 11:37

      Are you one and the same?

      Delete
    5. Anon @ 12:55: Yes

      Delete
    6. But I'm sure if a rodeo clown acts out somewhere across these United States you'll be right on it.

      Delete
  2. dear mr. somerby,

    i hope this letter finds you doing well. looks like winter is coming early here. hope its not too very cold where you are.

    i come to your site for your thoughts on the media in general but i find your columns on how education is treated in the media difficult to follow bc i dont know that much about educational testing and im not really that interested in learning about the subject. maybe im a bad person for admitting this, but i find it the subject dreadfully dull.

    on the other hand, labor issues are probably even worse treated in the mass media than education -- to the extent labor is dealt with at all in the business dominated corporate media. its less arcane and i would think more people could relate to these issues than to education...not too say that educational are not also important.

    you declare yourself a liberal so i would assume you mean by that...that you value the rights of labor and the economic issues which effect the vast majority of americans. in that spirit why not give some focus to the way the labor movement is covered and not covered when it should be in the media? the proposed trade deal between the u.s a. and the the european union might be a good place to start.

    Best regards,

    b4p

    ReplyDelete
    Replies
    1. somerby happens to have a background in education so he can comment meaningfully on it

      Delete
  3. Where is Finland? I can't seem to locate it on a map.

    ReplyDelete
  4. Really fascinating post. Esp. the actual appears accustomed to assistance the top. I'd not have access to observed this by myself. Will need to appear very carefully from aged pictures to determine this particular function.

    League of Legends boost

    Cheap Fifa 14 Ultimate Team Coins

    ReplyDelete