Regarding Finland and Massachusetts, watch the Atlantic move script: We hadn’t planned to comment further today about the reporting of the new PISA scores.
That said, Motoko Rich’s report in the New York Times turns out to be utterly woeful. Since we quoted so much of her report in our earlier post, we thought we ought to provide a bit of clarity.
At the Atlantic, Julia Ryan makes her usual succession of errors concerning the new PISA scores. That said, she offers a more coherent framework than Rich for discussing the scores.
Ryan reports, or attempts to report, where the United States ranked among the 34 OECD nations which took the test. Her focus on those 34 nations makes much more sense than Rich’s framework in the Times.
Thirty-one other nations and education systems took part in the PISA testing, but many of them are Third World nations. There is little point in reporting the fact that the U.S. surpassed their performance.
That said, that’s the approach Rich took in today’s New York Times. Earlier, we posted her findings, which were accurate but largely unhelpful.
The simplest way to summarize the U.S. performance involves that core 34. This is where the U.S. ranked among the 34 OECD nations:
United States rank among OECD nations, 2012 PISAThat represents the U.S. rank among those 34 nations. In terms of average scores, the U.S. essentially scored right at the OECD average in science and reading, somewhat below the average in math.
In reading: 17th out of 34
In math: 26th out of 34
In science: 21st out of 34
Again, that represents the American standing among the 34 OECD nations. That said, several of the highest scorers come from outside the OECD fold.
In math, the four highest scorers were non-OECD entities—Shanghai, Singapore, Hong Kong, Taiwan. These entities all outscored the U.S. by large margins.
As we noted earlier today, the U.S. tends to do better on the TIMSS, a conventional test of knowledge of curriculum. The U.S. tends to do less well on the PISA, which brands itself as a measure of “critical thinking.”
Different people have different ideas about which test may be more significant. We’d be inclined to lean toward the TIMSS, but we offer no firm preference.
In this morning’s New York Times, Rich’s report was woeful. Ryan’s report in the Atlantic is almost as bad.
This is the way Ryan starts. The errors start right away:
RYAN (12/3/13): The U.S. education system is mediocre compared to the rest of the world, according to an international ranking of OECD countries.Does the PISA “focus largely on math?” Ryan doesn’t seem to know how the PISA works.
More than half a million 15-year-olds around the world took the Program for International Student Assessment in 2012. The test, which is administered every three years and focuses largely on math, but includes minor sections in science and reading, is often used as a snapshot of the global state of education. The results, published today, show the U.S. trailing behind educational powerhouses like Korea and Finland.
Every three years, the PISA tests students in all three subjects—reading, math and science. That said, the point of emphasis rotates every three years.
The focus was on math last year, in 2012. But the focus was on reading in 2009, on science in 2006.
Ryan is straight out of college, with no particular background in education. She has bungled a succession of reports about testing in the past few months. (This report is her best.)
A publication like the Atlantic shouldn’t have such an inexperienced education reporter—but we rather plainly live in a post-journalistic world.
We note that Ryan still refers to Finland as an “educational powerhouse.” In math, Finland finished sixth among the 34 OECD nations, but its average score isn’t especially close to the aforementioned Asian tigers.
We’ll return to Finland’s performance below. As Ryan continues, a second error occurs. Note the self-contradiction:
RYAN (continuing directly): Not much has changed since 2000, when the U.S. scored along the OECD average in every subject: This year, the U.S. scores below average in math and ranks 17th among the 34 OECD countries. It scores close to the OECD average in science and reading and ranks 21st in science and 17th in reading.In math, does the U.S. rank 17th or 26th among the 34 OECD nations? Within the space of three paragraphs, Ryan states it both ways.
Here are some other takeaways from the report:
America Is Struggling at Math
The U.S. scored below the PISA math mean and ranks 26th out of the 34 OECD countries. The U.S. math score is not statistically different than the following countries: Norway, Portugal, Italy, Spain, Russian Federation, Slovak Republic, Lithuania, Sweden, and Hungary.
Alas, the answer is 26th. A full eight hours after posting, no one at the Atlantic has noticed this contradiction.
(For all PISA scores, start here.)
Might we note one further passage? Massachusetts took part in the PISA as an independent entity. Ryan scalds the state for its woeful performance:
RYAN: Even the top students in the United States are behind: This year, the PISA report offered regional scores for Massachusetts, Connecticut, and Florida. Massachusetts, which is a high-achieving U.S. state and which averaged above the national PISA score, is still two years of formal schooling behind Shanghai.Massachusetts is two years behind Shanghai! That sounds extremely gloomy. Aside from the fact that Shanghai remains a special case within China, this is what has been omitted:
That statement is only true about math, a point Ryan doesn’t make clear—and Massachusetts’ score in math is indistinguishable from Finland’s! That score bestowed “powerhouse” status on the Finns—and it earned our Bay State kids a good swift kick in the keister!
Regarding Finland and Massachusetts, their scores are indistinguishable in two of the three subjects tested. Massachusetts was a few points ahead of Finland in reading, a few point behind in math.
Finland outscored Massachusetts by 18 points in science, less than half a year according to the PISA's 39-point rule of thumb. Shanghai outscored Finland by a larger margin, some 35 points.
Let’s note the power of script:
Massachusetts matched Finland in reading and math, was somewhat behind in science. But on the basis of those three scores, Finland was called an “educational powerhouse,” Massachusetts was sent to the doghouse. That’s what happens when under-qualified reporters try to maintain well-known scripts.
Ryan shouldn’t be working this beat for a journal like the Atlantic; she simply doesn’t have the background. But, as we have often told you, we are plainly living in a post-journalistic world.
This is important and I am going thoroughly through the report and awaiting your comments.
ReplyDelete"That statement is only true about math, a point Ryan doesn’t make clear—and Massachusetts’ score in math is indistinguishable from Finland’s! That score bestowed “powerhouse” status on the Finns—and it earned our Bay State kids a good swift kick in the keister!"
ReplyDeleteHah!
Oh, we're DONE with Finland. Bunch of losers, those kids. This week we want to be JUST LIKE Shanghai.
When does the TIMSS come out? I need a heads up for when we again find and crown The Smartest Kids In The World. Speaking of smart kids, Amanda Ripley was the celebrity edu-expert who opined on the scores at the USDOE meeting. That's truly frightening. Can they not find someone who actually knows something?
If the PISA is testing on different curriculum than the TIMMS, it should be no surprise that the scores are not the same. Nor should the various countries score the same on either test if their local curriculums vary in content. If our students were going to work in one of these other countries, it might matter whether they are ahead or behind the students there, but they are going to work in the US and in US companies. Presumably our curriculum has been designed to meet the needs of our society and to prepare students for lives here. I'm not sure what the value is of making these kinds of comparisons with other countries.
ReplyDeleteBob, thank so much for teaching me how to read this data.
ReplyDeleteThere are at least two important questions here: (1) How are the PISA or other test scores to be interpreted? (2) Would it be wise to design curriculum so as to maximize our students' scores in, say, the PISA testing? Mr. Somerby has been focusing on the first question. Diane Ravitch's blog has been focusing on the second as well. The MSM, predictably, comes off badly on both counts.
ReplyDeleteThe next TIMSS assessment is 2015. The results will be released in 2016.
ReplyDeleteoakley sunglasses cheap, polo ralph lauren outlet, christian louboutin, burberry outlet, prada handbags, michael kors outlet, longchamp outlet, gucci outlet, coach outlet store online, kate spade handbags, louboutin, michael kors outlet, louis vuitton handbags, michael kors outlet, louis vuitton, coach outlet, burberry outlet, louis vuitton outlet, ray ban sunglasses, coach purses, nike shoes, michael kors outlet, polo ralph lauren, tory burch outlet, longchamp handbags, true religion jeans, louboutin shoes, louboutin outlet, air max, tiffany and co, louis vuitton outlet, ray ban sunglasses, louis vuitton outlet stores, jordan shoes, coach factory outlet, prada outlet, nike free, true religion jeans, tiffany and co, longchamp handbags, air max, michael kors outlet, oakley sunglasses, oakley sunglasses, michael kors outlet, chanel handbags, kate spade outlet
ReplyDeletenike free, nike roshe run, ray ban sunglasses, north face, ralph lauren, michael kors pas cher, michael kors, air max, mulberry, lululemon, hollister, nike blazer, nike air max, true religion outlet, polo ralph lauren, louboutin, north face, converse pas cher, hollister, ray ban pas cher, new balance pas cher, sac hermes, air max, true religion outlet, nike tn, barbour, hogan outlet, nike trainers, polo lacoste, vanessa bruno, air jordan, sac guess, abercrombie and fitch, sac louis vuitton, louis vuitton uk, oakley pas cher, vans pas cher, sac longchamp, louis vuitton, nike free pas cher, nike roshe, sac louis vuitton, michael kors, timberland, longchamp, air force, air max pas cher, sac burberry
ReplyDeletereebok outlet, jimmy choo outlet, canada goose, new balance shoes, ghd, beats by dre, bottega veneta, ugg boots, celine handbags, north face outlet, instyler, valentino shoes, uggs outlet, nfl jerseys, birkin bag, soccer jerseys, asics running shoes, insanity workout, hollister, marc jacobs, mont blanc, soccer shoes, mac cosmetics, babyliss pro, ugg boots, rolex watches, chi flat iron, canada goose outlet, ugg, vans shoes, mcm handbags, abercrombie and fitch, p90x, nike roshe run, canada goose jackets, wedding dresses, giuseppe zanotti, ugg pas cher, canada goose, uggs outlet, north face jackets, ugg australia, nike huarache, longchamp, ugg boots, herve leger, ferragamo shoes, lululemon outlet
ReplyDeletemoncler, hollister, ray ban, karen millen, pandora charms, links of london, air max, converse shoes, wedding dresses, montre homme, pandora charms, lancel, louboutin, vans, parajumpers, thomas sabo, converse, ralph lauren, canada goose, baseball bats, oakley, hollister clothing store, hollister, pandora jewelry, canada goose, swarovski, moncler, rolex watches, iphone 6 cases, moncler, canada goose, coach outlet store online, juicy couture outlet, air max, swarovski crystal, moncler outlet, toms shoes, moncler, juicy couture outlet, moncler, moncler, timberland boots, canada goose uk, louis vuitton, supra shoes, ugg, gucci, ugg
ReplyDeletenike shoes, Cheap Jordans,Cheap Jordan Shoes,Cheap Air Max,Cheap Free Run Shoes,nike shoes,nike outlet,nike factory,nike store,nike factory outlet,nike outlet store,cheap nike shoes,nike sneakers, toms outlet, toms outlet,tom shoes,toms shoes outlet,tom shoes,toms wedges,cheap toms,toms.com, air jordan, air jordan,jordan shoes,cheap jordans,air jordans,jordan retro,air jordan shoes,jordans,jordan 11,jordan xx9,jordan 6,new jordans,retro jordans,jordan retro 11,jordan 5,air jordan 11,jordans for sale,jordan 4,jordan 1,jordan future,jordan 3,jordan 12,michael jordan shoes,air jordan retro, cheap jordans, cheap jordans,cheap jordan shoes,cheap jordan,cheap jordans for sale,jordans for cheap,jordan shoes,jordans,air jordan,jordan retro,jordan 11,jordan xx9,jordan 6,new jordans,air jordans,retro jordans,jordan retro 11,jordan 5,air jordan 11,jordans for sale,jordan 4,jordan 1,jordan future,jordan 3,jordan 12,michael jordan shoes,air jordan shoes,air jordan retro, jordan retro, jordan retro,jordan shoes,air jordan,air jordans,retro jordans,air jordan retro,jordans,jordan 11,jordan xx9,jordan 6,new jordans,cheap jordans,jordan retro 11,jordan 5,air jordan 11,jordans for sale,jordan 4,jordan 1,jordan future,jordan 3,jordan 12,michael jordan shoes,air jordan shoes, air max 90, air max 90,nike air max 90,air max 95,air max 2014,air max 2013,air max 1,nike air max,air max,nike air max 2014,airmax,nike air max 2013, air max 95, air max 95,nike air max 95,air max 90,nike air max 90,air max 2013,nike air max,air max,air max 2014,nike air max 2014,airmax,nike air max 2013, nike free 5.0, nike free 5.0, nike free trainer 5.0,nike free run 5.0,free running 2,nike free run,nike free,free running,nike running shoes,nike free trainer,free runs,free run 5.0, omega watches, omega watches,omega watch,replica watches,rolex watches,replica omega watches,rolex,watches for men,watches for women,rolex watches for sale,rolex replica,rolex watch,cartier watches,rolex submariner,fake rolex,rolex replica watches,replica rolex
ReplyDeleteyoga pants, yoga pants,lululemon,lululemon outlet,lululemon athletica,lululemon addict,lulu lemon,lulu.com,lululemon.com, beats by dre, beats by dre,beats headphones,beats audio,beats by dr dre,beats by dre headphones,dr dre,dre beats,beats by dr,dr dre beats,dre headphones,beats by dr. dre,cheap beats, ferragamo, ferragamo,salvatore ferragamo,ferragamo shoes,ferragamo outlet,salvatore ferragamo outlet,ferragamo belts,ferragamo belt,ferragamo outlet, nike blazer, nike blazer,blazer nike,nike blazer pas cher,Chaussures Nike Blazer,Nike Blazer Femme, nike air force, nike air force,air force,nike air force 1,nike air force one,air force one nike,air force 1,air force one, michael kors outlet, michael kors outlet,michael kors handbags,michael kors uk,michael kors,michael kors,michael kors bags, cheap jerseys, cheap jerseys,nfl jerseys,cheap nfl jerseys,jerseys from china,cheap jerseys from china, burberry uk, burberry, timberland boots uk, timberland
ReplyDeletePorno, Porno izle, TĂĽrk Porno
ReplyDeletePorno, Porno izle, SikiĹź izle
Porno, Porno izle, SikiĹź izle, SikiĹź
Porno, Porno izle, SikiĹź izle, SikiĹź
Porno, Porno izle, SikiĹź izle, TĂĽrk porno
Porno izle, Porno, SikiĹź izle
Porno, Sex, Porno izle
Porno, Porno izle
Porno SikiĹź, Porno, Porno izle
Porno, Porno izle, SikiĹź izle
Porno, Porno Sex SikiĹź, Porno izle
Porno, Sikiş izle, Türk Porno, Kızlık Bozma
Porno, Sikişme izle, Türk Porno, Kızlık Bozma
TĂĽrk Porno izle, TĂĽrk Pornosu, TĂĽrk Sex, TĂĽrk SikiĹź
Porno Film izle, TĂĽrk Porno, SikiĹź
Porno izle, Porno video seyret
TĂĽrk Porno Resim, TĂĽrk Porno, TĂĽrk Porno izle, TĂĽrk sikiĹź,
ReplyDeletePorno, Porno izle, SikiĹź izle, TĂĽrk porno,Mobil Porno,
Porno, Porno izle, SikiĹź izle, TĂĽrk porno,Mobil Porno,
Porno, Porno izle, SikiĹź izle, SikiĹź, TĂĽrk porno, Mobil Porno,
Porno, Sikişme izle, Türk Porno, Kızlık Bozma, Porno izle,sikiş,
Porno izle, Porno video seyret, TĂĽrk porno ve sikiĹź seyret, Porno,
Türk Porno izle, Türk Pornosu, Türk Sex, Türk Sikiş,Türkçe Sex,
TĂĽrk Porno, TĂĽrk Pornosu, TĂĽrk SikiĹź, TĂĽrk porno izle, Porno izle, Porno,
Porno, SikiĹź, Porno izle, Mobil Porno, TĂĽrk Porno, SikiĹź izle, Seks izle