CASE STUDY: Only 16 (white) students believed the false claim!

TUESDAY, DECEMBER 27, 2022

Or quite possibly, none of them did: We'll repeat our earlier confession:

When we read the claim in the Washington Post, we didn't necessarily believe it.

We'd seen too many bogus claims from our own blue tribe in the past! That includes many claims like the claim in question—claims which express our blue tribe's love for "the racialization of everything."

Below, you see the claim in question. When we saw it in real time, we didn't assume it was accurate:

NORRIS (12/9/20): We are not just tussling with historical wrongs. A recent study of White medical students found that half believed that Black patients had a higher tolerance for pain and were more likely to prescribe inadequate medical treatment as a result.

The statement came from this column by Michele Norris—a good, decent person who has spent several decades at NPR and at the Washington Post.

The statement was made by an upper-end journalist—but was the statement accurate? Was it really true?

Did half of a group of (white) medical students really believe that black patients have a higher tolerance for pain than their white counterparts? According to Norris, a recent study had shown that very thing—but was her statement true?

We did what we've done a million times—we clicked the link provided by Norris and set out in search of the study. You can see the study here—and we weren't wholly surprised when found that Noris's statement was just ginormously wrong

As we noted in the last episode of our own case study, 222 medical students and fourth-year residents took part in the UVa study. The numbers broke down as shown:

Number of participants in the study:
First-year students: 63
Second-year students: 72
Third-year students: 59
(Fourth-year) residents: 28

Total participants: 222

Those 222 medical students (and residents) took part in the now-famous study. Needless to say, all these participants were white white white white white.

According to Norris, half those white participants said they believed the false claim to which she referred. Spoiler alert:

In fact, the study actually claims that only 16 of the 222 white participants said that they believed the claim—although, quite possibly, none of them actually did!

How could an upper-end journalist like Norris have been so grossly mistaken? In part, it seems that a long, sad game of "Telephone" led to her groaning misstatement that day. 

We'll examine the slippery writing involved in that hapless game when we file our next report in this series. For today, we'll merely show you how many participants are reported by the study's authors to have said that they believed the false claim—and we'll explain why it's possible that none of them actually did.

Let's take a look at the numbers! If you examine the study in question, you'll see a breakdown of the participants who (allegedly) said they believed the false claim. 

(Warning! The numbers which appear in the relevant chart represent percentages, not numbers of participants. The study's authors make that fact clear, but their layout could be a bit tricky.)

 Let's start with the fourth-year residents:

Of the 28 fourth-year residents, only one (1) is recorded as having said that he believed the false claim. Only one, out of 28!

The third-year medical students were even more skeptical! As you can see in the study itself, none (0) of those 59 participants are recorded as having said that they believed the false claim.

We've now accounted for 87 of the 222 participants. For the record, these are the 87 participants who were farthest along in their medical studies at the time in question.

Of those 87 participants, only one (1) is recorded as having said he or she believed the claim! And as we'll explain below, it's entirely possible that he or she never said any such thing!

Mathematically, one out of 87 is a great deal less than half! That said, here's the complete breakdown of the number of participants who (allegedly) said that they believed the false claim:

Number of participants who (allegedly) said that they believed the false claim:
First-year students: 5 of 63
Second-year students: 10 of 72
Third-year students: 0 of 59
Fourth-year residents: 1 of 28 
Total: 16 of 222

According to the authors of the actual study, 16 of the 222 participants said they believed the false statement. In a column in the Washington Post which remains uncorrected to this very day, Norris said that half these sketchy (white) medical students made this racist statement.

For the record, 16 out of 222 is 7.2%. Mathematically, that's substantially less than half. 

If we restrict ourselves to the third- and fourth-year participants—the participants who had already had two or three years of medical training—only 1 out of 87 stood charged with believing the false claim. That would be 1.1% of the group, an extremely small percentage.

Let's review:

According to Norris, 222 medical students participated in the study. These students were all white white white—and half of them said that they believed the false claim.

The actual text of the actual study doesn't make that claim. The study says the actual number is 7.2%.

That said, the study almost surely overstates the number of participants who said they believed the false claim. In fact, it's entirely possible that none of the (white) medical students said they believed the false claim. 

As we explained in our last report, here's the reason why we say that:

In the course of this study, the participants were asked to react to fifteen different statements about biological racial differences. Four of the statements were true; eleven of the statements were false.

The procedure employed in the study strikes us as slightly odd. As we explained in our last report, here's the way the process worked:

After being shown a statement, participants were asked to select from six possible reactions. They weren't allowed to say, "I don't know." Their only choices were these:

Possible reactions to each statement:
Definitely untrue 
Probably untrue
Possibly untrue
Possibly true
Probably true
Definitely true

Participants weren't given the option of saying they didn't know. If they didn't have any real idea if a statement was true or false, they had to select one of two (equivalent) reactions: 

They had to say it was "possibly true" or that it was "possibly untrue."

On its face, that strikes us as a slightly odd procedure. As we explained in our last report, a second procedure strikes us as flatly deceptive:

If participants chose "possibly true" as their assessment, they were scored as believing the statement was true—as "endorsing" the statement!

Langston Hughes said he'd seen rivers; to this day, we believe him. For ourselves, we've seen lots of thumbs on lots of scales as tribunes from our own tribe's academic and journalistic elites strain to establish claims reflecting preferred Storyline.

Let's think what that last procedure means, returning to the data concerning the third- and fourth-year participants.

According to the study, 87 such people—all of them white!—took part in the study. 

According to the study, only one of the 87 said he believed the false statement to which Norris referred. In fact, that one white person may have said that the statement was "possibly true."  

In the real world, that doesn't mean that he believed the statement in question. In a world where scholars put fists on scales, it's taken to mean that he did!

Your lizard brain will get busy now, insisting that we're somehow being unfair to someone. Like Ben Johnson in The Last Picture Show, we've been putting up with your lizard's trashy behavior all through our long life!

Bottom line for today:

Norris said, in the Washington Post, that half the medical students said they believed the false statement. (All the students were white!) In fact, only 16 out of 222 may have said that they believed the false statement—and it's entirely possible that none of the students did.

Norris' error remains uncorrected. Within our hapless, self-impressed blue tribe, blacks and whites join hands together to tell us the stories we'll like!

Coming next: Irate, slippery constructions—and a game of Telephone tag

25 comments:

  1. There is no point in repeating the reasons why this is specious on Somerby's part, even though Somerby keeps repeating his nonsense, not just day after day, but year after year.

    Let's think instead about the impact of 16 medical students graduating from their training without correcting their beliefs about black people. Each of those 16 physicians will go on to treat hundreds of patients, systematically underestimating the pain of black people in need, and perhaps committing errors that threaten their health. And for no reason other than our society's bigotry, that permitted them to complete training with such mistaken beliefs. Because their training programs had the same attitude as Somerby -- well 16 mistaken doctors can't be that bad! Would you want to be the person who encountered such a doctor?

    And let's think about the remedy to such mistaken beliefs. It is simply training about the actual needs of black people, to correct such mistakes. Can our medical training programs really not spare the time to do that? Or is Somerby arguing that there are no mistaken beliefs? The study does not show that at all.

    And why does Somerby tend to dismiss these findings, despite their importance for some number of black people? Why does he work so hard to convince anyone that the study is wrong? What motivates someone to argue that correcting mistaken beliefs among med students is not worthwhile? What meanness of spirit would cause a white person to argue that a study showing mistaken beliefs must be disregarded? And if that is not Somerby's intent, what is his motive? Is he trying to get that journalist fired?

    ReplyDelete
    Replies
    1. 16 students believed one false statement. How many believed each of the other false statements? How many does that add up to? And how many didn't know the true statements? And what about the finding that beliefing in those wrong statements led to underestimation of pain, a racial bias? That was the main point of the study, not the survey that was used to identify students with mistaken beliefs. Why does Somerby consistently ignore the second part of this study, which was the main point? And if the survey did not identify biased students, why were their judgments different when they looked at black targets compared to white ones? And they were different.

      This is why Somerby's critique is bunk. He ignores the main part of the study and tries to discredit the findings using one single question and the confusing remark of a journalist. Fortunately, science doesn't work that way.

      Delete
  2. "Norris' error remains uncorrected."

    As mh pointed out, Norris did not make an error. She quoted a fact from the study. Somerby has misapplied it, ignoring what the study itself said. Norris did change the wording of the study slightly, to relate it to the overall issue of undertreatment for pain, instead of the more specific terms used by the study, which might mean less to a TV audience. Somerby is the one who does not understand this study (or is misrepresenting it in order to claim it is wrong). Somerby appears to want to discredit the idea that there is bias in medicine. That is a hopeless task given the many other studies that supplement this one, all showing the existence of racial bias. I listed some of them back in 2020, when Somerby first talked about this particular study. Maybe Somerby hopes his readers will have forgotten that discussion, so he can spread his disinformation anew. Given that lives may be at stake, this is a very crappy thing to do.

    ReplyDelete
  3. "According to the study, 87 such people—all of them white!—took part in the study. "

    This is untrue. The results for the non-white participants are described in a different part of the paper. Somerby doesn't know how to read research papers, or he deliberately didn't tell his readers about the full results. I quoted the non-white results from the paper a week or so ago.

    ReplyDelete
    Replies
    1. Somerby's error remains uncorrected...

      Delete
  4. Researchers don't mind-read their subjects. If a subject endorses a false statement, they aren't allowed to decide that maybe the subject was confused, or didn't actually believe the false statement, or would have answered a different question differently. Data is data and you don't get to change it just because you don't like it or suspect it may be wrong (you redo the study in that case).

    ReplyDelete
  5. "Like Ben Johnson in The Last Picture Show, we've been putting up with your lizard's trashy behavior all through our long life!"

    I would say that we've been putting up with Somerby's trashy lizard long enough!

    ReplyDelete
  6. Michele Norris has a high tolerance for baloney.

    ReplyDelete
    Replies
    1. Neither you nor Somerby has shown that the study is baloney.

      Delete
    2. Possibly true, possibly untrue.

      Delete
  7. This comment has been removed by the author.

    ReplyDelete
  8. Two of the choices, "Possibly untrue" and "Possibly true", MEAN THE SAME THING! IMO choosing either of those is equivalent to "I don't know."

    ReplyDelete
    Replies
    1. What matters is not your opinion but what people actually mean when they use those options. That is an empirical question -- one that was explored in research described in that list of books I posted last week about judgments under uncertainty. Science is not done by forming opinions and assuming they are true. It is done by exploring behavior under controlled conditions.

      Delete
    2. David’s statement is possibly true, possibly untrue.

      Delete
  9. I gave up on this about halfway thru Bob's
    second day of obsession with it (you are not going
    to get anything but redundancy in these cases),
    but when Bob gets to the "lizard brain" and missing
    the point of The Last Picture Show (he's talking about
    a group of mean teenage boys, bored trouble
    makers who have their clearest parallel in the
    Oath Keepers and Proud Boys) you know his
    argument is weak, he knows it too, and can only
    move on to invective.
    Anyway, this is another example of Bob as
    childish southern boy who won't take any guff
    off a damn yankee. You just have to let these
    things pass.
    Speaking of the Post, I notice that Mo Dowd
    seems to have moved onto writing about movie
    stars (if only She had recognized her true vocation
    in the first place) but the Times actually lets her turn
    her space over to her dullard, Trump voting brother.
    Sort of thing that once would have been fertile
    ground for Bob, before he went out of this mind.

    ReplyDelete
  10. I read this study. My skepticism of its validity was increased, if anything. Two major flaws I see (among several), that TDH also pointed out are these: (1) They lump together "definitely true", "probably true" and "possibly true (and the same 3 variables for "untrue"). Thus, if a respondent said it was "possibly true" that a black felt less pain, that is categorized identically with "definitely" or "probably" true. You can't tell how many of those (and it was way less than 50%) wrongly stated that blacks had less sensitive nerve endings. (2) categorizing someone who said it's possibly true that blacks are less sensitive to pain together with those (if any) who said this was definitely or probably true distorts the English language. Saying that something is "possibly false" is closer to saying something is "probably true" than saying that something is "possibly true." If I'm not sure that blacks feel less pain, I don't think it's true, but that it is "possible", I can't rule it out entirely, I'd say the statement is "possibly true." On the other hand, if I thought blacks were probably less susceptible to pain, but wasn't sure, I'd acknowledge that it is possibly false that that they are more susceptible. I could list numerous other flaws. [caveat - I can't rule out that I have somehow missed something in the report or got it wrong - anyone is welcome to show me how]. It seems this report is being used to prove the trope that our doctors, in significant measure are racist. I think such a conclusion is nonsensical.

    ReplyDelete
    Replies
    1. So, you are doubling down on Somerby’s mistaken criticism. If these things were flaws, don’t you think that would have been noticed during peer review? You are both ignorant fools. You make no mention of the judgement task, which suggests you didn’t read the whole study.

      Delete
    2. “It seems this report is being used to prove the trope that our doctors, in significant measure are racist. “

      AC, The study says this: “Racial bias in perceptions of pain (and possibly treatment) does not appear to be borne out of racist attitudes. In other words, it is likely not the result of racist individuals acting in racist ways. To date, then, it is unclear what beliefs account for disparities in pain assessment and treatment.”

      My opinion is that the reports are merely trying to call attention to a problem, that black patients are not always treated properly for pain due to misconceptions. Why is this a controversial result? And why must you take it to the Fox News reduction ad absurdum, which is “liberals say that white med students and white doctors are all a bunch of racists.” The point is to improve medical care.

      Delete
    3. anon 6:42 and MH - I notice that neither of you refuted what I said. It's fine to try to improve medical care, but the report seems biased. Also, MH you ignore that the study is being cited as proof that the medical system is racist, and that blacks are understandably justified in avoiding covid shots - even though the study doesn't purport to stand for that. Plus TDH is taking, as usual, all this abuse for making legitimate points.

      Delete
    4. I explained and answered Somerby’s comments a year ago and also last week. I get it that you didn’t understand but that doesn’t mean I didn’t answer. And no, Somerby has no legitimate points. He doesn’t know what he is talking about and neither do you.

      Delete
    5. Good points AC/MA. The limited options and the wording of the study are absurd. Reminds me of that other study Somerby picked apart a few months ago -- the one that said something like "white people today are responsible for racism in the past," or some such nonsense, and if you answered disagreed with the statement, that counted as a racist answer (essentially). These researchers almost seem to be deliberately designing these reports in such a way that will produce a certain "shocking" result -- wow, look at the high percentage of racists! Good on Somerby for calling them out.

      Delete
    6. AC, the contention that the study is being used to somehow show there is racism in medicine may or may not be true, but that is not the fault of the study, which explicitly says (as I quoted) that it is not calling the bias a result of racism. It’s right there in the study. As for Somerby taking abuse, well, he is the one who is not simply accusing the reporting about the study of being used to prove “racism”, but calls the study itself into question.

      Delete
    7. anon 10:29, since you post as "anon" there is no way to tell what you post. I did read something last week that was probably you, which I admit I didn't understand because it was gibberish. You don't refute what TDH points out (me and anyone with critical faculties can see the same thing) because you apparently can't. MH, you still won't or can't refute the flaws in the study that I refer to; like Mike L observes, the study is absurd.

      Delete
  11. My own final thoughts on this: Do I believe a study might be
    more or less rigged to reach a conclusion that can't really
    be backed up? Yes. Obviously in this case one poster
    strongly disagrees, and I also believe it's possible he is
    correct. Do I believe Bob has become a kind of reverse
    racism crank? Yes. The lizard brain stuff underlines this.
    Based on my own medical procedures of recent years,
    I would strongly suggest to ANYONE undergoing said to
    make sure you know what your Doctor is going to do
    to control pain you might experience. I do believe that
    they are now careful, perhaps too careful, not to get
    get you hooked on pain killers. Make sure your Doctor
    is mindful of your wishes.

    ReplyDelete