CASE STUDY: What did "white medical students" believe?

THURSDAY, DECEMBER 22, 2022

A slightly odd research design: Readers, can we believe the things we're told by our most trusted tribunes? By the most trusted tribunes from our own blue tribe?

Sadly, we think the answer is no. In the case study we continue today, we offer one rather strange example. 

At stake is a widely stated claim about the offensive and racist beliefs of "white medical students." In Tuesday's report, we showed you three instances in which some version of this claim has been stated in the Washington Post or in the New York Times. 

In our experience, the claim in question is stated on a fairly regular basis. In Tuesday's report, one example came from the Washington Post's Michele Norris, a well-known former NPR anchor and also a good, decent person:

NORRIS (12/9/20): We are not just tussling with historical wrongs. A recent study of White medical students found that half believed that Black patients had a higher tolerance for pain and were more likely to prescribe inadequate medical treatment as a result.

Norris is a good, decent person with a long career in high-end mainstream journalism. As vaccine resistance grew within the black community, that was her account of the reason why many black Americans don't trust the medical establishment.

We're willing to admit it! When we read the highlighted claim, we didn't assume it was accurate. We didn't necessarily believe that half of a group of white medical students had said they believed that black patients have a higher tolerance for pain. 

We didn't automatically believe it! We'd encountered too many bogus claims down through the years—bogus claims which pleasingly reinforced our blue tribe's preferred Storylines. 

We decided to take a look at the recent study to which Norris referred. Today, we'll show you some of the things which struck us as strange about that widely cited study, the text of which you can peruse right here.

The 222 participants:

As you can see at that link, 222 medical students participated in the part of the study under review. According to the text of the study, their numbers broke down like this:

"first years, n = 63; second years, n = 72; third years, n = 59; residents, n = 28."

In short, participants included 194 people who were still in medical school, plus 28 (fourth year) residents. 

All the respondents were "white." There was no attempt to evaluate the beliefs of any other group of medical students.

The 15 statements at issue:

In the part of the study under review, the 222 medical students (and residents) were asked to evaluate a set of fifteen statements. According to the authors of the study, eleven of the statements are false. Four of the statements are true.

These are the fifteen statements respondents were asked to assess:

  1) Blacks age more slowly than whites
  2) Blacks’ nerve endings are less sensitive than whites’
  3) Black people's blood coagulates more quickly than whites'
  4) Whites have larger brains than blacks
  5) Whites are less susceptible to heart disease than blacks*
  6) Blacks are less likely to contract spinal cord diseases*
  7) Whites have a better sense of hearing compared with blacks
  8) Blacks’ skin is thicker than whites’
  9) Blacks have denser, stronger bones than whites*
10) Blacks have a more sensitive sense of smell than whites
11) Whites have a more efficient respiratory system than blacks
12) Black couples are significantly more fertile than white couples
13) Whites are less likely to have a stroke than blacks*
14) Blacks are better at detecting movement than whites
15) Blacks have stronger immune systems than whites

In the view of the study's authors, the four statements bearing asterisks are true. The other eleven are false.

The six permitted assessments:

Ther 222 medical students (including residents) were asked to assess each of those fifteen statements. They weren't asked to state their view of the various statements. Instead, they were given a list of six possible responses.

At this point, we begin wonder about the design of this study. The six responses available to the participants are listed here:

Definitely untrue 
Probably untrue
Possibly untrue
Possibly true
Probably true
Definitely true

Participants were asked to assess each of the statements in one of those six ways. Perhaps there's something we don't understand about some aspect of survey design, but we note one point of puzzlement:

There is no apparent difference between two of those permitted responses! If you say that a statement is "possibly true," you're automatically saying that it's also "possibly untrue." It seems odd to us to offer six possible assessments, two of which seem to be essentially equivalent.

Maybe there's something we don't understand about this type of survey design. We'll admit that we wondered how the study would have turned out if respondents had instead been given these five choices:

Definitely untrue 
Probably untrue
I don't know
Probably true
Definitely true

How would the study have turned out then? We have no way of knowing, though we could offer a guess.

The way those responses were scored:

We've shown you the fifteen statements respondents were asked to evaluate. We've shown you the six possible assessments they were allowed to make.

According to Norris, half the respondents (something like 111 out of 222) said they believed the second statement listed above—said they believed, in her paraphrase, that "Black patients had a higher tolerance for pain."

As we'll show you tomorrow, that statement by Norris was blatantly, grossly inaccurate. For today, we'll end by showing you the strangest part of this research design—an apparent part of this study's design which strikes us as truly remarkable.

Respondents were given six different ways to "score" each of the fifteen statements. Did respondents believe the various statements, eleven of which were false?

Astoundingly, this seems to be the way the authors of the research "scored" the respondents' assessments. Once again, we're offering text from the study itself:

We collected data from a total of 418 medical students and residents. Two hundred twenty-two met the same a priori criteria as in study 1 and completed the study (first years, n = 63; second years, n = 72; third years, n = 59; residents, n = 28)...On average, participants endorsed 11.55% (SD = 17.38) of the false beliefs. About 50% reported that at least one of the false belief items was possibly, probably, or definitely true.

Are we reading that correctly? That passage seems to suggest that, if a respondent rated a statement as "possibly true"—which means that it's also possibly false—the respondent was recorded as "believing" the statement. 

That strikes us as a very strange procedure. This further except from the text of the study further suggests that this actually was the procedure:

For ease of interpretation and ease of presentation, we collapsed the scale and coded responses marked as possibly, probably, or definitely untrue as 0 and possibly, probably, or definitely true, as 1, resulting in percentages of individuals who endorsed each item. 

According to that language, if a respondent said that some statement was "possibly true," that was taken to mean that the respondent had "endorsed" the statement in question.

That strikes us as a strange procedure—perhaps as "ease of interpretation" gone wild. Consider:

In the rapidly shrinking real world, if someone says a statement might be true, does that mean the person believes the statement? Does that mean that the person has somehow "endorsed" the statement?

That strikes us as a strange type of scoring on the part of the study's authors. In part, we say that for this reason:

According to the authors of the study, four of the fifteen statements in question actually are true. For example, this statement is said to true:

"Blacks have denser, stronger bones than whites."

Is that statement actually true? For ourselves, we don't have the slightest idea! Neither, we're willing to guess, did quite a few of the medical students and residents who took part in this study.

We'll also guess they had no idea about some of the other statements. That includes some of the eleven statements which are said to be false. 

That said, the design of the study gave them no way to say they simply didn't know if these statements were true. 

If they didn't know if a statement was true, they had to check one of the two (equivalent) assessments saying the statement was "possibly" true or untrue—and if they were unlucky enough to check the box marked "possibly true," they were apparently scored as believing / endorsing the statement in question.

To all appearances, this seems to be the way participants' responses were scored. This seems to mean that we have no real way to know how many of the respondents actually did believe the various untrue statements, including the untrue statement Norris cited in the Washington Post.

That said, we do know this:

We do know that Norris' statement in the Post was grossly, wildly inaccurate. In fact, nothing even dimly resembling half of the study's participants checked a response endorsing that statement in any conceivable way.

Michele Norris is a good, decent person and an experienced, high-ranking journalist. Given prevailing blue tribe Storyline, her claim about those white medical students was vastly pleasing.

Her claim was also grossly inaccurate. As you can see from some of the text we've posted above, it wasn't even an accurate statement of what the study's authors had said.

Norris' claim was grossly inaccurate. It remains uncorrected today.

Tomorrow: A look at the actual numbers


76 comments:


  1. Déjà vu, dear Bob. Didn't you already express your indignation at this good-decent smear... eh, sorry: this good-decent bullshit study, a while ago?

    ...okay, whatever...

    ReplyDelete
  2. Two points to make here:

    1. Somerby is apparently trying to bufoonishly imply that the study is trying to show how white medical students are a bunch of racists. The authors of the study make clear that they are not attempting to show anything at all about racism. The intent of the study is to identify gaps in knowledge about black patients. The medical students need to know what is and isn’t true in order to properly diagnose and treat their black patients.
    2. Despite Somerby’s harping on the idea that race is a construct, and chastising liberals for continuing to talk about “race”, it is clear from this study that blacks do have differences that may determine the proper diagnosis and treatment. (Has Somerby heard of sickle cell anemia?) Thus, while clearly race should not be used to deny opportunities, it clearly is a factor in proper medical care.

    ReplyDelete
    Replies
    1. "race" is not a factor in medical care, dear mh. Genetics is.

      Delete
    2. mh is a good, decent person, but the study was garbage.

      Delete
    3. @11:46 -- Cecelia "forgot" to use her nym again.

      Delete
    4. MH you are completely avoiding the point. This one study has been seized upon to show that doctors have unconscious racism in treating blacks, and justifies blacks in their reluctance to get Covid shots - something different from what you say is the purpose of the study.

      Delete
    5. Anonymouse 12:00pm, I did not write it and I never forget to use my name, the system occasionally forgets me.

      Delete
    6. This study contains two parts. Somerby only tells you about the first part, the questions used to identify students with mistaken beliefs. He does not tell you about the other part of the study, which shows that those with mistaken beliefs make different ratings of white versus black "target" patients. That is the part that demonstrates the "unconscious racism".

      The conclusion that blacks may avoid doctors and shots because of racist treatment in the past is based on many past atrocities that have contributed to widespread attitudes in the black community about avoiding white doctors due to possible bad treatment. This study demonstrates that bias exists, so that black concerns are not all in their heads, as Somerby likes to claim.

      Why do you suppose Somerby doesn't mention all parts of the study? Go look at the study for yourself and see what it includes, beyond what Somerby describes. Some of it answers Somerby's objections. But more importantly, it provides evidence to support those claims of bias, beyond just the wording of the questions used to identify those subjects who held more wrong beliefs.

      If Somerby's complaints were valid and the questions didn't identify biased subjects, then there would not be as strong a finding of bias as appears in the graphs showing bias. Inclusion of non-biased subjects in the biased sample would result in smaller effect, perhaps even a non-significant one. So even if you admit all of Somerby's criticisms to be valid, excluding those subjects with "possible" responses would make the bias stronger not make it disappear, because only the most clearly mistaken subjects would be responding in the task.

      Delete
    7. Cecelia said: "I never forget to use my name, the system occasionally forgets me."

      In other words, any anonymous statement may possibly be written by her, or may possibly not be hers.

      Delete
    8. Nope. I always inform other posters when this has happened and that what was written was by me.

      Delete
    9. This is from the study:

      “Racial bias in perceptions of pain (and possibly treatment) does not appear to be borne out of racist attitudes. In other words, it is likely not the result of racist individuals acting in racist ways. To date, then, it is unclear what beliefs account for disparities in pain assessment and treatment. Here, we examine the extent to which beliefs about biological differences between blacks and whites (e.g., beliefs that blacks have thicker skin than do white people or that black people’s blood coagulates more quickly than white people’s blood) are associated with racial bias in pain perception and treatment recommendations.”

      Delete
    10. According to this view, we must understand Greek tragedy as the Dionysian chorus, which always disburdens itself anew in an Apollonian world of pictures. The choric parts, therefore, with which tragedy is interlaced, are in a manner the mother-womb of the entire so-called dialogue, that is, of the whole stage-world, of the drama proper. In several successive outbursts does this primordial basis of tragedy beam forth the vision of the drama, which is a dream-phenomenon throughout, and, as such, epic in character: on the other hand, however, as objectivation of a[Pg 69] Dionysian state, it does not represent the Apollonian redemption in appearance, but, conversely, the dissolution of the individual and his unification with primordial existence.

      Delete
    11. The post labeled “mh” at 3:45pm is not mine. This is a big no-no, but I’m not surprised some troll is doing this. Do you see why people are hesitant to post as a nym?

      Delete
    12. No. The reason people are hesitant to get a nym is that they don’t want to be accountable as to what they say from day to day.

      You can get a verified nym.

      They also know that a board full of anonymous posts makes a comment board look flat and uninteresting.

      Delete
    13. I have been unable for about a year to log into Google in blogger to use my blogger id, which is also mh. And yet, I still label my comments. And some douchebag comes along and abuses it. Maybe it was you, Cecelia?

      Delete
    14. mh, maybe it was your mom. She thinks you should spend more time cleaning your room.

      Delete
    15. Anonymouse 4:37pm, wait till anonymices accuse you of not correctly spelling your name.

      Delete
    16. mh, your problem may be that you’ve restricted sites from tracking your internet activity.

      Delete
    17. Cecelia, mh makes the most cogent comments of anyone here, so calling mh a teenager is moronic. The most objectionable thing about the conservative commenters here is their gratuitous hostility, which you have just illustrated again.

      You make yourself especially foolish when you try to tell other people to use nyms, as if it makes any difference at all to the quality of discussion here. With or without a nym, you are an awful person and it is what you say that makes that clear, not your name.

      Delete
    18. Anonymouse 5:49pm, if you ever bothered to address what someone actually says, let alone recognize context, then you’d acknowledge that I’ve been responding to accusations.

      No, you’re going to distort what others say, then knock down your own distortion, then label your contrarians as being bad people.

      You’re a bore.

      Delete
    19. mh is polite until provoked. You come here aggressive and nasty. I point that out and you call me a bunch of names. Typical.

      Delete
    20. Cecelia, the painful fact that Bob seems never to want to face about organic existence is that everything dies. Humans know we are part of the natural world, and yet we are compelled to renounce it because it disagrees with our needs for endurance, safety, and control. In this post, Somerby points out that this dispute between our acceptance and our denial of our organic nature is the reason that the media have simultaneously employed both empathy and abstraction in their news articles and cable talk news shows. When that resembles what we know objective life to be, Republicans are projecting a physical attachment to the universe. Conversely, when Democrats vote to send billions of our tax dollars into a perpetual war machine, we are projecting a psychic attitude toward the cosmos.

      We should get together sometime you and I, little lady. What do you say?

      Delete
    21. You had me at “organic existence”.

      Delete
    22. Fuck off, losers. I’m back to my Google login. Assholes.

      Delete
    23. So allowing tracking worked then?

      Delete
    24. mh, I get you back with your Google logon after you couldn’t log in for a year, and this is your way of saying thanks.

      Sad.

      Delete
    25. In times like this I like to get naked and rub nougat all over my flaccid white flesh, every square inch, including my man boobs. Then I play my Quarterflash CD and read me some Lawyers, Guns and Money on the Ipad right there in the tub.

      Delete
    26. The best thing about the Democratic party, which I love, is its support for our Perpetual War Machine and empire. It was so so great when we Democrats voted last week to spend tens and tens of billions of dollars to continue that greatest of American sports: war and death.

      This blog however, is a problem. A BIG problem.

      Delete
    27. Cecelia, call off the freak who keeps using mh’s nym.

      Delete
    28. I love to bathe with men.

      Delete
    29. Anonymouse 11;07pm, I helped mh get back into his google logon after a year and he came barreling back to berate me. You give him a hand.

      Delete
    30. Cecelia says, "if you ever bothered to address what someone actually says, let alone recognize context...'

      It's no wonder people believe every Right-wing accusation is really a confession.

      Delete
    31. mh at 9:43,
      Agree that only the Left has agency. It, along with the fact that bigotry isn't my number one go to, is the reason I would never under ANY circumstance vote for a Republican for political office.

      Delete
  3. The woods are lovely, dark, and deep. Doctors are ethically obliged to care for all patients, even Russian trolls.

    ReplyDelete
  4. "A slightly odd research design"

    Somerby has no training in research design and wouldn't know whether this design is odd or not.

    ReplyDelete
    Replies
    1. It sure ain’t even.

      Delete
    2. Ha ha ha ha, you made a funny. Black people in pain is so hilarious.

      Maybe you want to agree with Somerby that the design was odd, but like him, you don't know what to say about the study that would discredit it. You keep on being your racist self. No one can force you to be a good, decent person when it comes to making medical treatment better for everyone.

      Delete
    3. Just get vaxed and boosted.

      Delete
  5. "We decided to take a look at the recent study to which Norris referred."

    Actually, Somerby first dissed this study back on December 20, 2021. Today, he makes it seem like he just looked it up, because of Norris's statements, but Somerby has been trying to discredit this study for a lot longer.

    This is one of the ways in which Somerby is not always perfectly honest in his essays here. I would say that statements like this verge on lies and are not honest at all, but your mileage may vary, as Somerby used to say, back when he was somewhat more honest.

    ReplyDelete
    Replies
    1. Correction, December 10, 2021

      "These (white) medical students today!"

      Delete
  6. You would think that Norris had conducted this study, because Somerby never mentions the actual authors. He does link to the study:

    Racial bias in pain assessment and treatment recommendations, and false beliefs about biological differences between blacks and whites

    Kelly M. Hoffman khoffman@virginia.edu, Sophie Trawalter, Jordan R. Axt, and M. Norman Oliver

    Proceedings of the National Academy of Sciences

    Somerby will not know this, but that is a prestigious place to be published. The National Academy of Sciences includes top researchers from various branches of science who serve as advisors to the President and our government. It is an honor to be selected to be part of that body -- the current members select future ones.

    If you consider Somerby's judgment about research design to be superior to that of the researchers themselves and the reviewers of the article prior to its publication, you are a moron.

    Somerby's ongoing attempts to undermine the concept of expertise leads to his assertion that his own beliefs about research are not only equally valid to those of trained scientists, but his assertion that his views are superior to those of these researchers and the reviewers of the study. Somerby doesn't know what he is talking about.

    He is, obviously, strongly motivated to deny the existence of racial bias in health care. How do we know that? He has been discussing this study and trying to discredit it, and Norris (and others) for more than a year now. If Somerby were to find a flaw in this study (all studies have flaws), it wouldn't overturn the conclusions -- that would require an assessment of the impact of the flaw on the findings. Somerby is unqualified to decide that too. But Somerby wants more -- he thinks any flaw he finds discredits the entire line of research and allows him to argue that claims about racism in medical treatment are overblown or even manufactured. Science doesn't work that way. To disprove findings, Somerby (or some actual researcher) would need to correct the perceived flaws and redo the study, obtaining different results that showed that the flaw had produced the previous wrong conclusions. Somerby is nowhere near being able to do such a thing. So, Somerby's carping has no ability to overturn this study.

    ReplyDelete
  7. "In short, participants included 194 people who were still in medical school, plus 28 (fourth year) residents."

    Somerby thinks this is a criticism. He is obviously unaware that medical school students have contact with patients. They participate in supervised clinical practice classes in which they interact with patients doing things like taking patient histories. At some medical schools, these courses are reserved for the final one or two years. At others, they occur during all four years. The inclusion of the residents permits a comparison of whether more training results in less of a tendency to believe the false statements.

    ReplyDelete
    Replies
    1. anon 12:07 - How do you transform TDH stating a basic fact about there being 194 participants still in medical school as an example of him criticizing the report? You're concocting this.

      Delete
    2. Somerby leaves out this part of the study, when he says that only white subjects were assessed (only white subjects are reported, but more were tested and mentioned by the study) and he claims that med students perhaps just didn't know the answers to the questions:

      "Of note, one could argue that first-year students’ lack of experience in medical training may be unduly influencing the treatment recommendation accuracy findings. However, the pattern of results does not change when removing first-year students [F(1,147) = 5.50, P = 0.020, η2G = 0.01 (target race × false beliefs interaction)].
      Readers may also be interested in analyses for nonwhite participants (Asian, n = 43; black, n = 21; biracial, n = 28; Hispanic/Latino, n = 11; other, n = 3). When examining the same models for nonwhites, there were no effects for pain ratings or treatment recommendation accuracy (P > 0.250)."

      Somerby is claiming that this isn't a matter of racial bias leading to wrong belief. He says maybe the med students were just ignorant. That isn't supported by this analysis from the study itself, examining that question. He is also claiming that excluding the uncertain beliefs would have been fairer, but he is wrong about that because bias has a stronger impact on people when they don't know things for a fact. When people lack knowledge they fall back on their biases, so those are exactly the cases where a racial bias effect would be most visible. Somerby says they should be excluded because, being students, they don't know the truth about black physiology. But that is exactly the point of the study -- that students are not getting the correct info and their biases lead them to make mistakes.

      And no, I am not concocting this. Unlike Somerby, I know something about this topic.

      Delete
    3. anon 1:00 - you go on and on in your obsessive and screwy efforts to discredit everything TDH says. I certainly don't have time to go into everything you say that is distorted. However, just as an example - TDH did not say that the "excluding beliefs would have been fairer." He says it would make more sense to have as an option "I don't know" rather than possibly true" or "possibly untrue." interestingly, it appears that the pollsters grouped together "definitely true, probably true and possibly true" all as equivalently indication belief in the truth of a statement is erroneous. Believing that a false implies that statement is "possibly false" that the statement is probably true. It appears TDH may be mistaken in saying that possibly true is equivalent to possibly false both say the same thing. When I say something is possibly true, I'm saying I doubt it's true but it might be; and vice versa, if I say something is "possibly false" I'm saying I doubt it's false but it might be. T

      Delete
    4. AC/MA, you are clearly both uninterested in this study's techniques and motivated to defend Somerby. Perhaps others will care about whether Somerby's criticisms have merit or not. You don't want to invest any mental effort in that task. That is your right.

      But, you don't get to claim that I am incorrect without coming up with answers to the things I've said. I explained why Somerby's proposal is wrong. Take it or leave it.

      Somerby is a standup comedian with no reason training (as is obvious from his complaints about studies here). You don't know me or my background, but I do have that training and I have fully explained why Somerby is wrong. You can believe who you want, but you cannot claim I am wrong without putting up some explanation yourself. I don't think you can do that, because like Somerby, you don't know anything about this topic.

      If you were given a decision to make and didn't know how to make it, you wouldn't say possibly false with consistency. Various things would affect your judgment, and I mentioned a few of them. One is the exact wording of the question. Another is your mood at the time you made the decision. Another is the context of the decision. When you don't know the answer, various other things affect your choice of a response. I guarantee this is as true for you as it is for all other human beings. Who discovered this? A guy who won the Nobel prize. But you want to insist you are different because Somerby says something you agree with? I find that incredibly stupid.

      Remedial reading:

      Judgments Under Uncertainty: Heuristics and Biases

      Heuristics and Biases: The Psychology of Intuitive Judgment

      Choices, Values, and Frames

      I've read these. You haven't. Decide what you want -- you are the one who will suffer the consequences of believing what Somerby peddles here to the unwary.

      The Essential Tversky

      Delete
    5. all very interesting anon 2:25, but you haven't refuted, or even addressed, anything I just said.

      Delete
    6. If you think I didn't address what you said, you are too stupid to talk to. Bye now.

      Delete
    7. AC/MA, stating the number os subjects is a criticism by Somerby because:

      1) Somerby complains that the students, being students, don’t know the right answers. That was addressed at length.
      2) Somerby complains that they were all white. They weren’t, as quoted from the study.
      3) Somerby implies that it was a small sample, even though the significance test showed a significant result. He implies this by itemizing the groups and noting, each time, that 28 residents were included. If the sample were too small there would be no significant result and the study would not have been published.

      Delete
    8. 9:32,
      AC/ MA is a lawyer, so he already has all the answers.

      Delete
  8. "According to the authors of the study, eleven of the statements are false. Four of the statements are true."

    Somerby several times mentions that the truth of the true statements and the falsity of the false statements is based on the view of the authors. The authors did not decide which of these were true or false. That determination is based on the medical literature, which the authors used to classify the statements as true or false. The medical literature consists of published studies comparing outcomes and treatments and physiological differences of black and white patients for various medical conditions. This is a subfield of medicine, with its own research base.

    By pretending that the authors of the study made up their own answers to their survey, Somerby shows his ignorance of how such research is done, and he pretends that the results are arbitrary or limited to the person beliefs of the study authors. That is not how this research was done. But Somerby is willing to deceive his readers to make points, counting on ignorance to excuse what appears to me to be a motivated deception about this study. And that is unfair to the authors and misleading to those who care about pain treatment and medical care.

    ReplyDelete
    Replies
    1. anon 12:14 - please quote where TDH says in hos post that the truth or falsity of the various statement was "based on the view of the authors." This is what you do all the time. You twist something that TDH says, and then go on and on alleging that something he didn't say is wrong, and intentionally deceptive.

      Delete
    2. 12:59,
      Bob should make his points, so Mao, Cecelia, MH and the Anonymous' don't have to guess what point Bob is trying to make.

      Delete
    3. "According to the authors of the study..."

      This is where he says it, as quoted above already.

      Here is another place where he says it:

      "In the view of the study's authors, the four statements bearing asterisks are true. The other eleven are false."

      "In the view of the study's authors..."

      This is clear. It means Somerby is saying that the authors decided "in their view" which were true and which were false.

      Later Somerby says: "They weren't asked to state their view of the various statements." As if this were a matter of opinion, with all opinions equally viable. It is a matter of fact, supported by evidence.

      Why were they not permitted to state their views? Because you cannot do math, apply statistics, to qualitative statements (sentences consisting of words conveying meaning). The assessment of such views would be more subjective than the ratings provided, from which the subjects themselves chose their answers.

      Delete
  9. "There is no apparent difference between two of those permitted responses! If you say that a statement is "possibly true," you're automatically saying that it's also "possibly untrue." It seems odd to us to offer six possible assessments, two of which seem to be essentially equivalent."

    Here is another aspect of Somerby's ignorance. It does make a difference whether the wording is "possibly true" versus "possibly untrue" and these are not processed by people in the same way. People have a bias that causes them to evaluate negative and positive statements differently. Showing this was part of Kahneman & Tversky's work on use of rating scale and judgment biases under uncertainty (when answers are unknown). The use of more options provides a finer discrimination of meaning than the use of fewer options (which would collapse these two distinct options into a single one). It would thus yield a less precise answer than Somerby's approach and eliminate the ability to see whether uncertainty was biased toward belief or against belief. Given that some of the statements are true and others are false, that is important to be able to see.

    Somerby thinks that researchers just make this stuff up as they go along. There is a body of research about how to devise scales to measure a variety of things. In this case, the measurement of attitudes and opinions consists of techniques which have been studied themselves, before they are applied to answer other questions, to see how scale use by a respondent might operate independent of the content of the question itself. Kahneman and Tversky wrote several books and Kahneman won a Nobel prize for his work on judgment. Researchers are trained in this stuff. Somerby thinks his own intuitive ideas about how to ask questions are just as good as decades of research about what people actually do when answering questions on surveys.

    There is another problem too. Somerby thinks that people can and should think logically -- that is, according to the formal rules of logic. Many cognitive studies have shown that people do not reason that way. Somerby's insistence that probably true and probably false mean the same thing is based on formal logic, where they would. Human beings are biased against loss, so probably false (negative) means something different to them than probably true (positive). That's why the glass half empty/half full task produces different responses in different people, and the answers can be shifted around by many incidental contextual cues, such as which option is first in order of appearance in the survey question, whether someone is in a bad mood, etc.

    If you don't have any idea how to do this kind of research correctly, as Somerby clearly does not, then it is possible to bias your results. Somerby seems to understand that, but he does not trust the researchers to know their jobs and he thinks his own untutored ideas are more valid than the scientists who did the study. What keeps those scientists honest? Peer review. Someone with expertise in scaling would have commented on the potential flaws and problems in how the questions were worded and scored, and the study would have been rejected had those been problematic enough to have affected the results. You expect peer reviewers to know these things because they spend many years in grad school and have research experience following that in doing exactly this kind of research. Somerby has none of that, not even the casual reading on the topic now available to those interested in cognitive science.

    ReplyDelete
  10. "According to Norris, half the respondents (something like 111 out of 222) said they believed the second statement listed above"

    Norris didn't say 111 out of 222. That is Somerby's addition. Norris is obviously taking her statement from the 50% stated by the researchers. Somerby has to put his thumb on the scales by putting words into Norris's mouth that she didn't say.

    This arises from another misconception by Somerby. There are 11 false statements out of 15, 4 true ones. The premise of the study is not that all biased med students will have the same misunderstandings of black physiology. It is that medical students do not know enough about black patients to treat them effectively, that they hold various sorts of misconceptions. That is why there are 15 questions that do not say the same thing. A student can get a wrong answer by failing to agree with the true statements, not just endorsing the false ones.

    Somerby's insistence that 111 out of 222 must get any every one of the statements wrong or any given one wrong is his idea of what would make the students deficient in their knowledge. The researchers didn't express their findings that way. Obviously, being wrong in belief about any of those statements could negatively affect treatment of a patient depending on their ailment, so insisting that all of the questions must be incorrect makes no sense. Any wrong belief can be damaging. The variety of wrong beliefs indicate the need for education, but also the tendency of an untrained public (as med students are when they enter training) to believe wrong things in general, although not all the same wrong belief, necessarily. There is not one single misconception, there are several, even many.

    Here is the conclusion Somerby quoted:

    "On average, participants endorsed 11.55% (SD = 17.38) of the false beliefs. About 50% reported that at least one of the false belief items was possibly, probably, or definitely true."

    This says that 50% of subjects endorsed at least one of the false beliefs. The study doesn't break down the % for each question, so Norris was incorrect when she generalized from the entire composite rating to one specific question. She was trying to make the rating concrete for viewers by substituting a single question for the larger concept of misbeliefs of all kinds used in the survey. But she is correct that 50% of the subjects held at least 1 mistaken belief. The average per false question was 11% endorsement. This is a tiny nitpick on Somerby's part, intended to portray the subjects as less biased than they were.

    To make this complaint, Somerby must ignore the graphs that relate the wrong beliefs to an actual underestimate of pain for a black "target" they were asked to assess, compared to a white "target". The subjects who believed more wrong things made different pain ratings than those who did not. And that is the heart of the study, not Somerby's quibble about how the wrong beliefs were described by Norris -- who gets the outcome of the study right.

    ReplyDelete
  11. The GOP has huge assholes.

    ReplyDelete
  12. "Definitely untrue
    Probably untrue
    I don't know
    Probably true
    Definitely true"

    Somerby suggests allowing a category of "I don't know" which he thinks would include both possibly true and possibly false responders. This would be a good idea if the point of the survey were to assess the knowledge of the med students. But that is not the purpose. The purpose is to assess bias in the form of mistaken beliefs about black people's medical needs.

    If you were to include the "I don't know" category, presumably most, if not all, of the students would select that option on most of the questions, and that would leave very few participants. It would complicate the data collection by requiring far more students be given the questionnaire in order to get large enough samples to do the task itself. Beyond that, the type of med student who might actually know the answers would be unusual in some way, not typical, and thus not a sample that could be generalized back to any larger group.

    If you want to show that bias leads to different judgments about medical needs, then you have to include biased students as well as non-biased ones. If you change the rating scale to exclude those who are biased, you cannot do the rest of the study. The approach used in this study is called "forced choice" because it requires a guess in the absence of knowledge by excluding "I don't know". It is a valid approach to studying biases, in situations where you cannot ask someone "Are you racially biased?" without getting "of course not" as an answer.

    ReplyDelete
  13. When a teacher tries to teach a student something and the student stubbornly refuses to learn, who suffers? Not the teacher.

    Somerby does not respond to his comments. It may be that he reads them, but if so, there is no evidence that he responds to anything anyone says there. It may be that he doesn't read them. In that case, he closes his mind and denies himself the opportunity to learn anything. When someone hears a message but refuses to respond to it in any way, that is considered a bad faith conversation, because it is one-sided.

    I don't know whether Somerby is discussing in bad faith, not discussing at all but propagandizing his readers, or not encountering his comments at all. But it doesn't matter to me. I can evaluate Somerby's arguments for myself. It does matter to Somerby, but he is the person who will have to continue living his life with his current views and information.

    I think the same is true of everyone here, from AC/MA and Cecelia, to myself, the other anonymous commenters and people such as mh, who often respond to Somerby's content. As some who values learning, Somerby often offends me. I also have difficulty understanding why someone would want to be wrong when they could be more knowledgeable. But it is each person's right to live and learn (or not) as they choose. But beliefs don't make facts. We have ways of knowing and testing our knowledge that help us separate facts from opinions and beliefs. Scientists tend to value the search for fact, knowledge of reality. Somerby, not so much. And that is another thing that offends me about his essays here. Reality doesn't change because of our beliefs. And reality has the ability to punish those who get things wrong, via consequences that are essentially impersonal but also inexorable. That's why it matters whether your beliefs conform to reality.

    Somerby's mistaken ideas and dishonest statements about this study will change nothing about whether doctors are biased and mistreat black patients. His views change nothing about the existence of racism, or even his own racism. His only achievement here is to confuse people, some more than others. Who benefits from that? Not the people he confuses. But perhaps those who make a living off of bilking the confused. If Somerby is paid for what he does here, he benefits financially. Or perhaps he does this to soften folks up for predators in the right wing, from Q-Anon to the Proud Boys and Trump and Trump-like grifters.

    ReplyDelete
    Replies
    1. If he offends you so much and is so ignorant, why is it you come here every day, numerous times, with verbose stream of consciousness twaddle?? (though as an anon there is no real way to know from anon post to anon post whether it's you or not)

      Delete
    2. To help you, but I see what a waste of time that is.

      Delete
  14. "Given prevailing blue tribe Storyline, her claim about those white medical students was vastly pleasing."

    No one on the left is pleased about racism, about black people enduring more pain because of racial biases and mistaken beliefs, or anything else that prevents all people from receiving appropriate medical treatment.

    Why does Somerby say horrible things like this, untrue things, about those on the left? Does it give him pleasure?

    ReplyDelete
    Replies
    1. Isn't this just more of the right wing's demonization of Democrats? An extension of the idea that liberals invented racism and are thus the ultimate racists?

      Delete
  15. If these examples of tbe blue tribe journalists
    Misleading people are so frequent, why does
    Bob have to milk one dubious one
    Into two long posts?

    ReplyDelete
    Replies
    1. two and counting...he isn't finished yet

      Delete
  16. Both statements are true.

    ReplyDelete
  17. Excellent work by Somerby. This is similar to his analysis of the logically flawed script that "election deniers are anti-democracy." Most of us just hear and read such scripts and never stop and think critically about them or go back to the original source as Somerby is doing with this study. It makes you wonder how much else we read/hear is flawed in some way. I'm glad someone is calling it out. I certainly would never do the kind of deep dives Somerby is doing. Life is too short!

    ReplyDelete
    Replies
    1. Oddly, Somerby describes only the beginning of the study and makes a few mistakes suggesting he only actually read the beginning of it. This is a pattern that also happened with the bio of Godel, the book on Einstein made simple, and even Chanel Miller’s book about her ordeal. I don’t think the phrase deep dive applies to Somerby at all.

      Delete
    2. I wonder if he knows how The Lady with the Lap Dog ends. He might not like it as much if he had ever finished reading it.

      Delete
    3. I can assure you, the election deniers I've heard complain about the 2020 Presidential election are not anti-democracy for white people.

      Delete