Part 2—Please come to Boston: How good are Finland’s miraculous schools?
In all such matters, it’s hard to say. It doesn’t help when our “educational experts” and our education reporters often seem to have so little sense of the way public schools really work.
(Case in point: The upbeat headlines on the upbeat report by Dana Goldstein in yesterday’s Slate. See below.)
It doesn’t help when our education journalists love the simple-minded “adventure stories” which give us so much narrative pleasure. Current example:
Miraculous Finland “rocketed from the bottom of the world to the top, without pausing for breath.” So Amanda Ripley says at the start of her widely-praised book, The Smartest Kids in the World.
Plainly, Finland didn’t do that. At this point, it isn’t even entirely clear that Finland is at the top of the world in the miraculous way we’ve all heard described over the past dozen years.
Case in point: the 2011 TIMSS. That’s the Trends in International Mathematics and Science Study, a major international test battery in which most developed nations have taken part at some point in the past two decades.
In 2000, Finland scored at the top of the world on the brand new PISA (the Program for International Student Assessment), a test of “critical thinking.” At that point, Finland stopped participating in the more conventional TIMSS.
Until 2011.
In 2011, Finland returned to the TIMSS. That same year, nine different American states participated in the TIMSS at the Grade 8 level as if they were independent entities.
(Only two states participated in that manner at the Grade 4 level.)
For that reason, the 2011 TIMSS produced average scores for Finland and the United States at the Grade 8 level. But it also produced statistically valid average scores for those nine American states.
No single test or test session should be regarded as definitive. But here are the average scores which resulted in Grade 8 math:
Average scores, 2011 TIMSSOn that particular test, six of the nine American states outscored miraculous Finland. Unless we want to be statistically silly, Florida tied the Finns.
Grade 8 math, all students
Massachusetts 561
Minnesota 545
North Carolina 537
Indiana 522
Colorado 518
Connecticut 518
[Finland 514]
Florida 513
[United States 509]
California 493
Alabama 466
(On the TIMSS scale, 500 is set at the international average, with a standard deviation of 100.)
Some of those score differences don’t amount to much. Colorado and Connecticut racked up four-point wins over Finland. Four points on the TIMSS scale isn’t a very big deal.
But when you look at those scores from the TIMSS, a question might pop into your head. Why are people flying to Finland to figure out how to run public schools? Why aren’t they taking Amtrak to Massachusetts instead?
Why aren’t they flying to Massachusetts? The question is strengthened by the Grade 8 science scores, on which Finland outscored the U.S. by a wider margin:
Average scores, 2011 TIMSSAs the poet so memorably pleaded, “Please come to Boston!” Why aren’t we hearing that song?
Grade 8 science, all students
Massachusetts 567
Minnesota 553
[Finland, 552]
Colorado 542
Indiana 533
Connecticut 532
North Carolina 532
Florida 530
[United States 525]
California 499
Alabama 485
Please note these key disclaimers:
We aren’t saying that Finland has crummy schools, since it surely doesn’t. We aren’t saying that Massachusetts has better schools.
We aren’t saying that you can sensibly reach sound judgments on the basis of any one test or test session. We aren’t saying the TIMSS is better than the PISA, on which Finland has tended to score better as compared to the United States.
We are saying this:
Data like these are part of the story, unless we’re just telling adventure stories or trying to reinforce the narratives preferred by our funders and sponsors. We’re also saying this:
These TIMSS scores were released on December 11, 2012. When her book appeared in August 2013, Ripley didn’t breathe a word about these scores. She simply continued churning the tale about the miracles worked in Finland.
On page 2 of her book, she grossly misdescribed her own chart about those alleged miracles, thus committing the largest misstatement in the history of books. As far as we know, no reviewer said a word about this ridiculous point.
Let’s be clear. Massachusetts didn’t score at the level of the Asian tigers. These are the scores of the top-scoring nations and entities on that TIMSS math test:
Average scores, 2011 TIMSSMassachusetts hasn’t rocketed to the top of the world. But it strongly outscored Finland in math, unless you read Ripley’s ballyhooed book and/or our post-journalistic “press corps,” which basically serves to keep repeating the scripts it receives from elites.
Grade 8 math, all students
South Korea 613
Singapore 611
Taiwan 609
Hong Kong 586
Japan 570
[Massachusetts 561]
[Minnesota 545]
Russia 539
In our view, there is more to learn, or to wonder about, within those 2011 TIMSS scores. We may get to that topic tomorrow, or we may have to wait until after Thanksgiving, which occurs on Thursday of this week.
This second aspect to those scores concerns our brutal American history. Finland is a middle-class, unicultural nation.
Our own striving nation is not.
Coming next: Disaggregation
Concerning Goldstein's report: At Slate, Dana Goldstein has presented an upbeat report about an upbeat educational study.
Here's our question:
At any point, in any way, did Goldstein consider the possibility that the teachers who produced the higher test scores in question might have accomplished that outcome by cheating? Did the experts who conducted the study protect against that possibility?
We first approached the Baltimore Sun about cheating on standardized tests in 1971. We did our first stand-alone column on that topic for the Sun in 1982.
Years later, we called our shot concerning Michelle Rhee’s absurd claims about her own greatness as a teacher. A few years later, additional evidence surfaced.
Earlier, we were skeptical about that feel-good story at the top of the Washington Post’s front page about the little low-income school with the wonderful test scores. As a result, we uncovered a statewide scam in the state of Virginia.
The Washington Post wouldn’t report that statewide scam even after the chairman of the state board acknowledged to us, to his great credit, that it had actually happened.
In recent years, cheating scandals have gotten so big that our major newspapers have sometimes reported them. But so what? The experts and the education reporters can’t get these basic facts of life into their office-bound heads.
Go ahead. Check that report! Did it even occur to Goldstein that some of those teachers might have been cheating?
We have no idea if that happened. But the question did enter our heads.
Why did the Loma Prieta earthquake get so much more news coverage than Hurricane Hugo, which occurred the same year and did a bit more damage? Maybe earthquakes are sexier than hurricanes. Maybe it was the link to the World Series. But, I also think reporters preferred to travel to San Francisco, rather than to coastal South Carolina.
ReplyDeleteI had the pleasure once of attending a convention in Helsinki, Finland. It's a lovely city. Maybe reporter like visiting Helsinki more than some of the how, humid Asian cities.
I dunno, the weather in Hong Kong can be pleasant in the fall and everyone speaks English. Also, you get more frequent flier miles and time to lounge on the plane in business class thinking about your next promotion.
ReplyDeleteDavid
ReplyDeleteThe driving idea behind the Education Reform movement is to destroy unions and monetize education.