One of our favorite pet peeves: It has long been one of our favorite pet peeves.
It’s also a problem which infests the journalistic treatment of academic studies, such as they are.
The problem doesn’t quite have a name. So let’s get started with a tease from Tuesday night’s All In.
At one point, Chris Hayes teased an upcoming segment. This is what he said:
HAYES (8/13/13): White people don’t like affirmative action unless they’re getting something out of it. I’ll explain coming up.Could that possibly be true? That’s what you might call a sweeping statement. But could it really be true?
Could it be true that “white people don’t like affirmative action unless they’re getting something out of it?”
Some white people don’t seem to “like affirmative action” at all! But Hayes seemed to be making a sweeping statement—a statement about all white people.
Could it possibly be true? That all “white people” feel that way about affirmative action?
It almost seemed that this might be the case based on Hayes’ next tease. Here’s what he said this time:
HAYES: Coming up, white people want college admissions to be a true meritocracy strictly based on test scores, unless they think white people don’t do well on those tests. New insight on white people problems, next.Interesting! Hayes was going to convey a “new insight on white people problems!” A few minutes later, this is the way his actual segment began:
“A jaw-dropping new study shows that white people don’t like affirmative action unless they think it’s going to benefit them.”
According to Hayes, the new study is “jaw-dropping.” Maybe this jaw-dropping study really does show that all white people think and feel this way about affirmative action!
In fact, that doesn’t seem to be what the jaw-dropping study actually shows at all. As he continued, Hayes described the study in a bit more, or perhaps a bit less, detail:
HAYES: So along comes this great experiment, a study by Frank Samson, a professor of sociology at the University of Miami. The study asked 599 white Californians how they felt about the importance of grade point average in the University of California admission system. Grade point average was generally rated as extremely important.Hayes is becoming a showman, as he explained to Salon a few weeks back. For that reason, you’ll have to forgive him if he goes on the air and pretends to be dumb, when he pretty much isn’t.
But half of the people in the study were randomly selected to receive this information. "Under current admissions procedures in the University of California system, Asians make up almost 40 percent of the student body or two out of every five students, while they are only 12 percent of the California population.”
Lo and behold, that group of white people who got that information gave grade point average less importance because suddenly they, white people, were the ones being threatened by the so-called meritocracy. Part of Professor Samson’s conclusion: "This finding weakens the argument that white commitment to meritocracy is purely based on principle."
In this case, note what that jaw-dropping study actually says. It says that white people “gave grade point average less importance” (our emphasis) when they were prompted to realize that strict reliance on GPA doesn’t advantage their group.
This isn’t exactly a study about attitudes toward affirmative action. More directly, it’s a study of attitudes towards what you might call “strict reliance on GPA.” But that isn’t our complaint about the way Hayes kept describing the study.
According to what Hayes says in that passage, this study found that white people are less likely to favor “strict reliance on GPA” when they come to see that this reliance doesn’t favor their group. The key word there is “less.”
The people in question were less likely to favor strict reliance on GPA. Until we get more information, this leaves us largely clueless.
Question: How much less likely are white people to favor strict reliance when they get that prompt? Hayes didn’t ask or answer that question at any point in the program. And when he brought on a famous professor as his guest, the famous professor didn’t inquire about this matter either.
It may be that white people are only slightly less likely to favor strict reliance. On the other hand, it may be that they’re much less likely.
But Hayes didn’t bother with any of that! He just took a vague formulation, then spoke of white people as a group, as if the study had found an attitude shared by all.
How much less likely are white people to favor “strict reliance?” If you actually care about this topic, you pretty much have to ask.
But Hayes didn’t ask that question; neither did the famous professor. As we’ve noted in the past, this sort of thing goes on all the time when journalists talk about studies.
Tomorrow: More from Hayes and the famous professor. Plus, a completely absurd report from—who else?—the New York Times.