Supplemental: Diane Rehm and the search for truth!

THURSDAY, AUGUST 21, 2014

Least likely guest on the planet: We’ll admit it—Amanda Ripley drives us a little bit nuts.

Last year, Ripley published a ballyhooed book about public schools, The Smartest Kids in the World. It was a gigantic favorite of all teacher-hating elites.

We’re willing to assume that Ripley’s intentions are good. But in some ways, we’d say that her book included the most cherry-picked facts on the planet. (Below, we’ll recall what she did.)

This Tuesday, we were listening to The Diane Rehm Show on our car radio. Like most people, we like Rehm. She previewed her opening hour:
REHM (8/19/14): Thanks for joining us. I'm Diane Rehm.

Americans have access to more news than ever before. But research shows we've never had less trust in our sources of information.

Joining me to talk about how to judge the credibility of news in the digital age, Alan Miller of The News Literacy Project; author and investigative journalist, Amanda Ripley; social media journalist, Andy Carvin of First Look Media; and Tom Rosenstiel, he's executive director of the American Press Institute.

I hope you'll join me. I know this will be an interesting and informative discussion.
Say what? For the full transcript, click here.

By rule of law, Rosenstiel has to be present for any such discussion. We’re not saying that’s wrong.

(We did the Rehm Show in October 2000; Rosenstiel was present then too. Frankly, we’re not sure he ever went home from the studio.)

If you plan to discuss “trust in our sources of information,” Rosenstiel has to be present. But why was Ripley there? Speaking of trust in sources of information, here’s what she did in her widely ballyhooed book:

There are two major sets of international tests for public school students—the TIMSS/PIRLS and the PISA. Almost all developed nations, including the U.S., participate in both sets of tests.

American kids tend to score better on the TIMSS/PIRLS. For that reason, advocates of “education reform” almost always cite the PISA.

The cherry-picking doesn’t stop there. They tend to cite the PISA math test, the subtest on which our kids have scored most poorly in recent administrations. (We've scored better in reading and science.)

In her widely cited book, Ripley cherry-picked those international data to death. In the course of writing an entire book, she never even mentioned the existence of the TIMSS/PIRLS tests.

You could read her entire book without being told that these tests exist! Like others who beat the drums for “reform,” she only mentioned the PISA.

She did make one exception. At one point, she cited data from the TIMSS to prove a few points about the alleged greatness of Minnesota’s reform-friendly schools—though she never mentioned where the data in question had come from. In short, good scores on the TIMSS showed that Minnesota was great. Good scores nationwide on the very same tests somehow got disappeared.

To us, that represented world-class cherry-picking of information. Two days ago, there she was, helping the nation answer Johnny Carson’s old question:

Crackers! Who do you trust? How do we know who we can trust for our sources of information?

Posted by on

48 comments:

  1. OMB ( Protecting America with the OTB)

    "American kids tend to score better on the TIMSS/PIRLS. For that reason, advocates of “education reform” almost always cite the PISA."
    BOB

    American kids tend to score better in lower grades. For that reason,
    advocates of the public school "status quo" always ignore high school NAEP.
    Zarkon

    Which nearly identical statement presents better evidence of fact?

    ReplyDelete

  2. I am trying to uderstand the basis for the claim that Ripley Cherry picks by using PISA instead of PIRLS and TIMMS.

    The premise of her book is that other countries' public school systems are producing better scoring students. Wouldn't you want to use the test that measured the kids when thye finished school?

    Not to make an inappropraite sports analogy, Mr. Somerby, but if you are comparing results of a race, wouldn't you want to measure who won at the end rather than how everyone is doing at the mid point.

    Frankly, if the US does better on tests for younger kids than other countries but falls behind among graduates or near graduates, I would say that is an even bigger indictment of US public schools and argument in favor of Ripley's point.

    ReplyDelete

  3. This doesn't address her failure to explain why she chose to focus on only one test when others exist. That explanation is especially important when looking at different tests presents a different picture, perhaps supporting very different conclusions about how kids are doing. Also the tests don't test the same things, which should be discussed, not ignored in her book.

    ReplyDelete
    Replies

    1. Sorry, my comment is responding to Anon@5:26, not Somerby.

      Delete

    2. 526 seems to have a dog in this hunt.

      Delete

    3. Her book was about high school students. Specifically the experience of three American exchange students. She focused on the international tests given to high school students.

      While your response is interesting it does not at all address my question. If American students do better on international tests at lower grades than they do at higher grades, doesn't that reinforce the point that the overall public school system in the US has a problem? Or conversely that other countries do a much better job as their students get older and finish the whole primary and secondary process? While Somerby may be right that Ripley should include more information about testing at intermediate levels of schooling, even if she did it would not detract from her overall premise, but instead reinforce it.

      Delete

    4. urban legend, I don't know if I have a dog or a platypus. But the first commenter aroused my curiosity about the logic of the whole post and testing.

      If you want to know if your schools are producing students capable of higher education, you should want to know and be concerned if lower grade testing showed problems, but you would be foolish to declare victory simply based on how elementary and middle school children perform.

      Delete

    5. She picked the test that produced results supporting her preferred narrative. There are complexities to comparing high school age kids related to changes in the population (drop outs in US, tracking in Europe and Asia, who is tested). This gets very complex but there is no reason to believe that the US does a worse job in the upper grades than in lower grades or that Poland is better.

      Space does not permit repeating the arguments discussed here when Somerby originally discussed Ripley's book but you can go back and read them. Or email Somerby directly.

      Comparing our high school kids to others in 3 randomly selected other countries doesn't tell us whether our kids ate prepared for college. How could it? We have better, more direct ways of assessing that

      Delete

    6. And yet, when talking about "remarkable gains," I recall a certain blogger using the test that best supports his preferred narrative, even going so far as to proclaim that one test alone as the "gold standard" of all testing.

      In short, the oft-used damning with faint praise, "Well, he may be full of shit and stuck in 1999 on politics, but he's still good on education" is losing its traction.

      Delete

    7. So your concern here is complaining about Somerby not discussing whether Ripley is the right person to be on this particular panel.

      Delete

    8. It might be. Or it might not be. It all depends on who you are asking. Care to share?

      Delete

    9. Remarkable gains undeed.

      http://www.huffingtonpost.com/2014/05/07/naep-2013-high-school_n_5276767.html


      Delete

  4. Yes @ 9:56 we do have ways of asessing that. And those assessments are not painting a picture which would argue for what the first commenter called "the status quo" approach.

    http://www.huffingtonpost.com/2014/08/20/act-scores-race_n_5693084.html

    ReplyDelete
    Replies

    1. Who exactly is arguing for the status quo approach? Not Somerby.

      Delete

    2. It is hard to tell what Somerby is "arguing for." He only "argues against."

      But he has even argued against establishing a national set of standards for elementary and secondary education, and even dipped into his personal insult bag to do a little chimpanzee-style hurling at the guy who had the temerity to fund the research to establish those standards.

      Delete

    3. That's because Bill Gates is no friend of public education.

      Delete

    4. Bob Somerby is the only true friend. He is also the only one who cares about black children.

      Delete

    5. Yes, 11:30. Despite the money he has poured into public education research, Gates is no friend of public education because no less authority than Somerby says so.

      After all, Somerby once failed as a teacher before he failed as a stand-up comedian.

      Delete

    6. Here is a quote from today's Inside Higher Ed:

      "Analysis shows deep-pocketed investors are searching for winners in an unsettled ed-tech market."

      Gates is concerned with developing education as a potential frontier for use of technology. He doesn't give a damn about kids. His idea of research is findings that support his agenda. He ignores research that shows that kids learn because of human contact with real-live teachers, which makes technology wrong as a primary delivery mode for learning.

      Many people say this -- not least Somerby. Research says this but Gates doesn't really like Research unless it supports his goals.

      After all, Gates once failed as a student before he became a robber baron of high-tech industry.

      Delete

    7. "After all, Gates once failed as a student . . ."

      Really? He scored 1590 out of 1600 on the SAT and was accepted into Harvard. He was so far ahead of the curve writing code as an undergraduate that even his professors admitted they had nothing to teach him, and encouraged him to leave to start his own company.

      Delete

    8. He quit Harvard to steal software and sell it under his own company. Getting rich isn't necessarily an indicator of success, except in business. He flunked ethics, obviously.

      Delete

  5. So Bob, what did Ripley have to say on the radio show? Or were you simply more interested in taking another whack at her book?

    ReplyDelete

  6. Just as with Social Security, there are elites who see no value in public education or any other "commons" for the greater public, that is unless they and their ilk are getting a big piece of the action. Beware whenever the wealthy and the well-connected speak of "reform". It's not a surprise our new fangled liberals have not caught on to this, and they probably never will.

    Ripley is a baby elite who will surely graduate to full fledged elite as long as she pleases the "privatization" skimmers and scammers.

    ReplyDelete
    Replies

    1. Your comment is a wastebasket of sloganeering gibberish.

      Delete

  7. I thought I asked a sensible question at 5:56. The blogger is self contradictory. His readers are mostly imbeciles.

    ReplyDelete
    Replies

    1. You asked an extremely sensible quesion, but unfortunately, you did not follow Somerby's preferred narrative when you "seemed" to discount the "remarkable gains" of our fourth graders, and asked whether that was as important as the final product, or "the end of the race" as your analogy put it.

      That will put you in a lot of trouble with the small remnant of Bobfans whose minds go all kablooey when logic is introduced into the discussion.

      Delete

    2. I didn't find myself in any particular trouble. The first response simply didn't answer my question but instead said I didn't address Somerby's complaint. I wasn't trying to do so, but in response I did, and also restated my question. The person called "urban legend" appeared to be making some kind of response, so I restated the question again. I never got a response, but the thread quickly took a direction I characterized late last night in a fairly ugly manner, which was perhaps too harsh. I also misstated the time of my first comment.

      I'll restate my qustion in a different way.

      If Ripley had used international test scores such as TIMMS and PIRLS in addition to PISA, wouldn't the fact that American students rank higher on tests at an earlier age but lower by the end of their public school careers have made Ripley's critique even more dramatic?

      Delete

    3. You should email BOB directly and you may get a response to your question. Why would you keep expecting a response out of a group of mostly imbeciles? What does that say about you?

      Delete

    4. I'll answer your question in a different way.

      NO.

      Even Ripley would understand that it is difficult to compare students at the end of their public school careers because of things like tracking and drop-out rates. That students appear to rank lower by the end of their public school careers may be an artifact of who gets tested, not how well students are doing. The only way to show that students decline in comparison to (1) other students in the US, or (2) students in other countries, would be to do longitudinal studies comparing students with their own previous performance. This is the same problem as with testing IQs over a span on years. The kinds of tests you would use at different ages vary and do not necessarily produce the same outcomes for the same kids because they test different things.

      If Ripley had used international test scores in addition to PISA it wouldn't necessarily have proven anything, much less something dramatic. If you are saying that Ripley could have cherry-picked for better support of her narrative by using such scores, consider that it would have been a lot more work for her to include those other tests and scores and would have perhaps confused her readers, so keeping her narrative simpler is more effective. When you start using statistics it is hard to say anything dramatic because people are turned off by math.

      So, the answer to your question is NO.

      Delete

    5. AnonymousAugust 22, 2014 at 10:25 AM -- I think you make a good point. I believe Bob doesn't read these comments. You could e-mail him at bobsomerby@hotmail.com

      BTW it would be helpful in these back and forth discussions if you chose a name.

      Delete

    6. I did choose a name! But I see your point. :)

      Delete

    7. Thank you for the effort @ 10:37, but your response adds nothing but more confusion.

      The thesis of this post by Somerby is that Ripley failed to use TIMSS and PIRL because they show US students doing better in comparison to other countries than PISA.
      I asked if showing those results would not reinforce her point.

      You state it would not because testing older students is problematic. So are you suggesting PISA should be abandoned and only TIMSS and PIRL should be used?

      You tehn wrote, if "Ripley had used international test scores in addition to PISA it wouldn't necessarily have proven anything."

      I would be tempted to ask at this point, then what the heck is Somerby complaining about.

      Delete

    8. David, just for the record, your last response addressed me.
      I have not chosen a name and I did not respond to you at 11:19.

      You state you don't beleive "Bob" reads these comment and suggest I e-mail him. So does another commenter.
      What is the basis for your belief? Why, if I have a question about something he posts in public on the internet, should he not answer in the same public forum?

      Delete

    9. 12:10 - He has responded to my emails before. Not always though.

      Delete

    10. Good to know 12:27, but not to the point. A blogger states things in a public forum. He should answer his readership there.

      Delete

    11. You should contact him directly because you asked something fairly technical that was off topic to the point made in the post.

      Delete

    12. 12:10 Bob has never, to my knowledge, responded to a comment on this board. In some cases, he has posted about something that had been commented on earlier. In these cases, he has never acknowledged the earlier comments. Bob has never amended a post in response to a comment pointing out an error. For these reasons, I don't think Bob reads the comments. Maybe he should read the comments, but that's a different question.

      OTOH he has sometimes replied to my e-mails, so I know he reads them.

      Delete

  8. 1:15

    You don't want an answer, you want an answer in the comment box. That doesn't make any sense to me so you are on your own there. "He should answer his readership there". According to what authority?

    Good luck with everything and have a good weekend.

    ReplyDelete

  13. nike shoes, Cheap Jordans,Cheap Jordan Shoes,Cheap Air Max,Cheap Free Run Shoes,nike shoes,nike outlet,nike factory,nike store,nike factory outlet,nike outlet store,cheap nike shoes,nike sneakers, toms outlet, toms outlet,tom shoes,toms shoes outlet,tom shoes,toms wedges,cheap toms,toms.com, air jordan, air jordan,jordan shoes,cheap jordans,air jordans,jordan retro,air jordan shoes,jordans,jordan 11,jordan xx9,jordan 6,new jordans,retro jordans,jordan retro 11,jordan 5,air jordan 11,jordans for sale,jordan 4,jordan 1,jordan future,jordan 3,jordan 12,michael jordan shoes,air jordan retro, cheap jordans, cheap jordans,cheap jordan shoes,cheap jordan,cheap jordans for sale,jordans for cheap,jordan shoes,jordans,air jordan,jordan retro,jordan 11,jordan xx9,jordan 6,new jordans,air jordans,retro jordans,jordan retro 11,jordan 5,air jordan 11,jordans for sale,jordan 4,jordan 1,jordan future,jordan 3,jordan 12,michael jordan shoes,air jordan shoes,air jordan retro, jordan retro, jordan retro,jordan shoes,air jordan,air jordans,retro jordans,air jordan retro,jordans,jordan 11,jordan xx9,jordan 6,new jordans,cheap jordans,jordan retro 11,jordan 5,air jordan 11,jordans for sale,jordan 4,jordan 1,jordan future,jordan 3,jordan 12,michael jordan shoes,air jordan shoes, air max 90, air max 90,nike air max 90,air max 95,air max 2014,air max 2013,air max 1,nike air max,air max,nike air max 2014,airmax,nike air max 2013, air max 95, air max 95,nike air max 95,air max 90,nike air max 90,air max 2013,nike air max,air max,air max 2014,nike air max 2014,airmax,nike air max 2013, nike free 5.0, nike free 5.0, nike free trainer 5.0,nike free run 5.0,free running 2,nike free run,nike free,free running,nike running shoes,nike free trainer,free runs,free run 5.0, omega watches, omega watches,omega watch,replica watches,rolex watches,replica omega watches,rolex,watches for men,watches for women,rolex watches for sale,rolex replica,rolex watch,cartier watches,rolex submariner,fake rolex,rolex replica watches,replica rolex

    ReplyDelete

  14. ralph lauren outlet, ralph lauren outlet,ralph lauren outlet online,polo ralph lauren outlet,polo ralph lauren outlet online,polo ralph lauren,ralph lauren,polo ralph,polo shirts,ralphlauren.com,polo outlet,ralph lauren polo, oakley sunglass, oakley sunglasses,cheap oakley,cheap oakley sunglasses,oakley sunglasses cheap,oakley outlet,oakley sunglasses outlet,oakley vault,oakleys,oakley.com,sunglasses outlet,cheap sunglasses,oakley prescription glasses,fake oakleys,oakley glasses,oakley store,fake oakley,oakley sale,cheap oakleys,discount oakley sunglasses, ray ban sunglasses, Ray Ban Sunglasses,Ray Ban Outlet,Ray Ban Sale,Cheap Ray Bans,Cheap Ray Ban Sunglasses,ray ban sunglasses outlet,ray ban,rayban,ray bans,ray-ban,raybans,ray ban wayfarer,ray-ban sunglasses,raybans.com,rayban sunglasses,cheap ray ban, burberry, burberry,burberry outlet,burberry outlet online,burberry factory outlet,burberry sale,burberry handbags, chanel bags, chanel bags,chanel handbags,chanel sunglasses,chanel outlet,chanel purses,chanel handbags official site, coach outlet store, coach outlet,coach outlet store,coach outlet store online,coach outlet stores,coach factory outlet,coach factory,coach factory online,coach factory outlet online,coach outlet online, chaussures louboutin, louboutin,louboutin pas cher,christian louboutin,louboutin chaussures,louboutin soldes,chaussure louboutin,chaussures louboutin,chaussure louboutin pas cher,louboutin france, sac michael kors, michael kors,sac michael kors,michael kors sac,michael kors pas cher,sac michael kors pas cher,michael kors france, north face outlet, north face outlet,the north face,north face,the north face outlet,north face jackets,north face jackets clearance,northface

    ReplyDelete

Subscribe to: Post Comments (Atom)