Statistics can be amazingly hard!

FRIDAY, MAY 14, 2021

Especially for upper-end journalists: Statistics are amazingly hard, especially for upper-end journalists.

For today, let's consider an error which doesn't matter. After that, let's consider an omission which pretty much does.

An error which doesn't much matter:

Yesterday, Jonathan Chait was having some fun with the latest report about Trump. Along the way, he made this obvious error:

CHAIT (5/13/21): We should pause the narrative to point out that getting somebody in Washington to call Donald Trump an idiot is almost a trivially simple task. This is in fact one of the most widely held opinions in the United States as a whole. (One poll found that 39 percent of Americans volunteered the description “idiot” for the President.)

We didn't even have to fact-check the highlighted claim. Even before we checked it out, it was obvious what Chait had done.

First things first! Chait provided a link in support of the highlighted claim. Unfortunately, it was a classic "link to nowhere." It took us to this report from May 2017—a report which doesn't make the highlighted claim.

That earlier report did identify the specific poll to which Chait was referring That earlier report included a classic "useless link"—a link to the Quinnipiac web site, not to the specific poll to which the report referred.

Still, we were able to locate the specific poll in question—this Quinnipiac poll from May 10, 2017. Obviously, that poll didn't "find that 39 percent of Americans volunteered the description 'idiot' for [Trump]."

Obviously, that didn't happen. Here's what happened instead:

Question #9 in that poll was a classic "first word that comes to mind" question. Here's the text  of the question, exactly as it appears in the Quinnipiac report about the poll's results:

9. What is the first word that comes to mind when you think of Donald Trump? (Numbers are not percentages. Figures show the number of times each response was given. This table reports only words that were mentioned at least five times.)

Which part of "numbers are not percentages" don't journalists understand? We ask because journalists constantly make the kind of mistake Chait made in this case.

As Quinnipiac reported the results of that survey question, it reported that 39 people had volunteered the word "idiot" when asked about Donald J. Trump. But that wasn't 39 percent of the survey's respondents. It was 39 people—39 people total.

For better or worse, Quinnipiac had interviewed 1,078 people in conducting this poll. That means that 3.6% of respondents had volunteered the word "idiot," not the larger 39 percent.

Why do public polling companies keep asking such "top of mind" questions? We have no idea. They always include the statement saying that the numbers posted "are not percentages," and journalists always respond by acting like they are.

It doesn't matter how often this mistake is made. The next time around, the mistake will be made again. Which part of the simple word "not" don't top scribes understand?

An omission which likely does matter:

Chait's mistake is a classic head-scratcher but no, it doesn't matter. An omission in a new report by the Washington Post pretty much does, unless we prefer unfettered Storyline to full-blown depictions of fact.

Yesterday, this new report appeared online. It hasn't yet appeared in print editions. Its dual headlines say this:

Police shootings of children spark new outcry, calls for training to deal with adolescents in crisis
A Washington Post database of fatal force incidents finds most children shot by police are minorities and less likely to be armed than adults shot by police

The report concerns the number of "children"—actually, people under age 18—who have been shot and killed by police officers since the start of 2015.

The report's data come from the Post's Fatal Force site. The report offers this overview:

KINDY ET AL (5/13/21): [Stavian Rodriguez] is one of 112 children who have been fatally shot by police between Jan. 1, 2015, and Monday, according to a Washington Post database that tracks fatal police shootings. Over the same period of time, 6,168 adults were shot by police.

[...]

The database shows that the circumstances leading up to the shootings of children are varied, with about half beginning with a robbery, a traffic stop, a stolen car or a 911 call. Most of the incidents took place during daytime hours; only one appears to have involved alcohol use by the child; and 19 of the children were experiencing a mental health crisis at the time of the shooting.

The database shows that children are frequently armed with a gun or knife during these fatal police encounters, but not as often as adults who die by police gunfire—63 percent of the time for children vs. 76 percent for adults.

Sixty-six percent of the children who died in fatal police shootings were Black, Latino, Asian or Native American compared to 44 percent of adults who were racial minorities.

Again, the word "children" in this report refers to anyone under age 18. Allowing for that bit of nomenclature, we'll focus on the (accurate) statement made right in the headline:

Most children shot by police are minorities.

Allowing for that bit of nomenclature, that statement is perfectly accurate. The specific percentage of non-white decedents is 66%, the Post's report says. That figure involves a minor statistical error on the part of the report's authors, but we'll call that number close enough for journalistic work.

It may or may not seem surprising to read that only 34% of under-18 decedents are white. At this point, though, we note a fairly basic statistical omission in the report—we note the fact that white kids are now a minority of the American public school population.

We'll guess that a lot of people are unaware that fact. This was the rundown from the National Center for Education Statistics as of September 2017:

NCES: In fall 2017, of the 50.7 million students enrolled in public elementary and secondary schools, 24.1 million were White, 7.7 million were Black, 13.6 million were Hispanic, 2.8 million were Asian/Pacific Islander (2.6 million were Asian and 185,000 were Pacific Islander), half a million were American Indian/Alaska Native, and 2 million were of two or more races. 

Between fall 2000 and fall 2017, the percentage of students who were White decreased from 61 to 48 percent, and the number of White students decreased from 28.9 million to 24.1 million.  Similarly, the percentage of students who were Black decreased from 17 to 15 percent, and the number of Black students decreased from 8.1 million to 7.7 million. 

In contrast, the percentage of students who were Hispanic increased from 16 to 27 percent during the same period, and the percentage of students who were Asian/Pacific Islander increased from 4 to 6 percent...

Between fall 2017 and fall 2029, the percentage of public elementary and secondary students who were White is projected to continue decreasing (from 48 to 44 percent)...

As of September 2017, the percentage of white public school students nationwide was 47.5 percent and slowly falling. 

It we use that as a starting point, the number of white youths shot and killed by police is still disproportionately low in simple statistical terms. But the disproportion may not be as large as people might be inclined to imagine or assume.

Why is there a disproportion at all? That would be a larger question, calling for careful analysis. For better or worse, we humans love our Storylines, even here in our self-impressed town.

Statistics can be very hard. They can also be misleading. 

It seems to us that the Washington Post made a significant omission in this new report. Just how large is the disproportion to which the Post may seem to be pointing? You'd think we'd all want to know that.

Where would an analysis go from there? In large part, that would depend on the extent of our love for preferred Storyline. In recent years, that love has been strong in Our Town.

That said, statistics can be amazingly hard, especially for upper-end journalists. According to international experts, Storyline is more pleasing by far. Our attraction to Storyline is hard-wired, these experts consistently tell us.

We'll say it again: Very few actual children get shot and killed by police. We'd like to see the Washington Post compare those Fatal Force numbers to the overall population of Americans aged 12-17, whatever those numbers may be.