STANDARD REPORTS ABOUT STANDARDIZED TESTS: American kids are the best in the world!

THURSDAY, DECEMBER 12, 2019

The problem we choose to ignore:
It would be hard to do!

It would be hard to overstate the amount of nonsense which appears when newspapers like the Post and the Times attempt to report, or pretend to report, on the state of our public schools.

Consider a bit more of what happened in last Tuesday's Washington Post. Results from the 2018 Pisa had just been released. In reading, American kids had outperformed their counterparts from such major "peer nations" as these:
Major nations the U.S. outperformed:
The United Kingdom, Germany, France, Spain, Italy, Russia; Japan, Taiwan; Australia
American kids had outperformed some very major nations! That said, they'd also outperformed their counterparts from a wide array of smaller boutique nations, including these:
Smaller nations the U.S. outperformed:
Denmark, Norway, Belgium, The Netherlands, Austria, Switzerland, many others
With statistical significance thrown into the mix, U.S. performance was indistinguishable from that of the Asian educational tiger South Korea. The American score was substantially higher than the average score for the 35 OECD nations.

American kids had outperformed the vast majority of the world's nations. But because the American kids had been outperformed by their counterparts in Macau and Estonia (combined population, less than 2 million), the Washington Post headlined, and led its news report, with this fantastical claim:

"Teenagers in the United States continue to lag behind their peers in East Asia and Europe in reading..."

So it said in the Washington Post! This claim was reproduced in a banner headline which ran across the top of last Tuesday's page A3.

Truly, that was a remarkable claim, but the absurdity was just getting started. Roughly fifty percent of the Post's report (ten paragraphs out of twenty) were devoted to an absurdly speculative "explanation" for the American kids' embarrassing failure on the Pisa.

What was wrong with these teens today? Balingit and Van Dam, and their unnamed editor, introduced this ridiculous theme right in their fourth paragraph:
BALINGIT AND VAN DAM (12/3/19): [T]he exam has faced a chorus of skeptics who caution reading too deeply in to the results. Students are not penalized for performing poorly and never see their results, and students in the United States tend to be less motivated to perform well on it compared with teens in other countries, according to recent studies.
Roughly half the Post report was devoted to "mounting evidence that the gap in scores between countries reflects a gap in effort as much as it does a gap in achievement. By both measures," Balingit and Van Dam reported, "the United States lags behind."

Paragraphs 4-5 and 13-20 were devoted to this new speculation, a speculation the Post attributed to "economists." In support of this idea, the Post included a large graphic designed to show that U.S. kids respond more favorably to financial incentives—to being paid to do well on a test—than their counterparts in Shanghai.

This was all designed to explain why U.S. teens had lagged behind their peers in reading—on a reading test where U.S. teens had outperformed the substantial bulk of the world.

This is how crazy the discourse can get when newspapers like the Washington Post attempt or pretend to report on American schools. On the front page of last Tuesday's New York Times, the reporting on these Pisa results was little better.

Tomorrow, we'll return to that Times report, noting a few of its oddities. For today, let's discuss the very large canine which didn't bark in either of these scripted "news reports."

For our money, American kids did amazingly well in both reading and science on these latest Pisa tests. We discussed our reasons for saying that in yesterday's report.

In both the Post and the Times, reporters noted an undesirable fact, if only in passing. In the Post's formulation, the new results from last year's tests "show widening disparities between high- and low-performing students in the United States, adding to a growing body of evidence showing worsening inequity in public schools."

This undesirable fact was mentioned by both the Post and the Times. Neither paper attempted to say how much wider the disparity was—how much the disparity had grown since the 2015 Pisa.

Still and all, the papers were displaying a minor ability to "disaggregate" the Pisa results—to compare the average scores attained by different groups of American kids.

For whatever reason, neither newspaper managed to perform a different, highly familiar form of "disaggregation." Neither newspaper reported what the Pisa scores look like when American results are disaggregated by ethnicity and race.

Dear friends, would it surprise you if you were told that two large groups of American kids would, if viewed as nations unto themselves, actually be the highest scoring nations in the world? Especially given what you read in the Post and the Times, would such information surprise you?

Would it surprise you if you were shown something like that? Because that's what happens when you disaggregate American scores by ethnicity and race on the Pisa reading test.

Below, you see some extremely basic data. We're including two basic parts of the data which weren't permitted to bark:
Average scores, Reading Literacy, 2018 Pisa:

Singapore: 549
Estonia: 523 (highest OECD nation)
Canada: 520
Finland: 520
(South) Korea: 514
United States: 505
United Kingdom: 504
Japan: 504
Australia: 503
Taiwan: 503
Germany: 498
France: 493

All OECD nations: 487

United States, white students: 531
United States, Asian-American students: 556
For all Pisa data, start here. More information below.

Should we count Singapore as an actual nation? Technically, we probably should.

That said, no other nation in the world outperformed tiny Estonia on the Pisa reading test. But white kids in the United States outscored the brainiac Estonians—and Asian-American kids blew their Estonian counterparts away.

Does it surprise you to know that our nation's ragtag collection of white kids would, if viewed as a separate nation, be the highest-scoring nation in the OECD, perhaps in the entire world? Please understand:

These aren't white kids from private schools, or white kids from high-income families.

These are American white kids across the board, including kids in Appalachia and the Rust Belt whose families are devastated by Oxycontin. That average score even includes the fumbling efforts of white kids in Mississippi and Alabama, on whom we all know to look down.

Even including all those kids, American white kids, viewed as a group, would be the highest-scoring nation in the world! And then, along come our Asian-American kids! They blow everyone out of the water, even their peers in Singapore.

These Asian-American kids today! They massively outperformed the ballyhooed kids of South Korea. They even outperformed the kids of Finland, darlings of the American press corps for the bulk of this century.

On the Pisa reading test, our Asian-American 15-year-olds blew the whole world away! That said, let's return to our white kids for a moment—the kids whose lack of effort allegedly matches their pitiful lack of smarts:

We all can see what those basic data show once they've been un-disappeared. Our white kids outscored brainiac Finland. They outscored the Asian tigers Korea and Japan by substantially more.

If they were viewed as a separate nation, they'd be the highest scoring full-blown nation on the face of the earth! Meanwhile, the data from the Pisa science test follow this general pattern:
Average scores, Science Literacy, 2018 Pisa:

Singapore: 551
Estonia: 530 (highest OECD nation)
Japan: 529
Finland: 522
(South) Korea: 519
Canada: 518
Taiwan: 516
United Kingdom: 505
Germany: 503
Australia: 503
United States: 502
France: 493

All OECD nations: 489

United States, white students: 529
United States, Asian-American students: 551
In terms of international ranking, the United States did slightly less well in science than in reading. But even here, American white kids scored one point behind highest-performing nation Estonia, while matching Japan and outscoring Korea. Meanwhile, Asian-American kids matched Singapore and obliterated the rest of the world.

Readers of the Post and the Times aren't allowed to know such things. Instead, they're told that U.S. kids are lagging the world, possibly due to lack of effort along with a lack of smarts.

Presumably, somebody in these major newsrooms knows how to "disaggregate" test scores. Everyone has seen it done, and everyone has heard about our punishing achievement gaps. There's nothing "new" about any of this. They just don't choose to do it.

That said, two large groups of American kids are the highest-performing in the world! Each group outperforms the ballyhooed kids of Finland, kids our "press corps" has long loved.

They outperform the exhausted kids of South Korea, who go to regular schools all day, then attend their workhouse-like academies every night. At the Washington Post and the New York Times, you not only aren't permitted to know this. You are instead deliberately given a vastly different impression.

The data we're showing you are remarkable, but you aren't encouraged to see them. The reason is fairly clear:

Those data define the modern-day version of "the problem we all live with"—and the horrible folk at the Post and the Times quite simply aren't willing to go there.

Tomorrow: The problem we all disappear

Regarding Pisa data: For all Pisa data, start here. In each subject area, U.S. scores are disaggregated by ethnicity and race under "Achievement by Student Groups."

The NCES makes a point of providing these important data. The Washington Post and the New York Times simply don't want to discuss them.

35 comments:

  1. Taegan Goddard (Political Wire) quotes Graff:

    "Garrett Graff: “The idea that Fox News represents a literal threat to our national security, on par with Russia’s Internet Research Agency or China’s Ministry of State Security, may seem like a dramatic overstatement of its own—and I, a paid contributor to its competitor CNN, may appear a biased voice anyway—but this week has made clear that, as we get deeper into the impeachment process and as the 2020 election approaches, Fox News is prepared to destroy America’s democratic traditions if it will help its most important and most dedicated daily viewer.”

    “The threat posed to our democracy by Fox News is multifaceted: First and most simply, it’s clearly advancing and giving voice to narratives and smears backed and imagined by our foreign adversaries. Second, its overheated and bombastic rhetoric is undermining America’s foundational ideals and the sense of fair play in politics. Third, its unique combination of lies and half-truths has built a virtual reality so complete that it leaves its viewers too misinformed to fulfill their most basic responsibilities as citizens to make informed choices about the direction of the country.”

    ---------------

    Today we hear that Russia funded Lev Parnas and Rudy Guiliani's efforts to smear Joe Biden via a Ukraine investigation instigated by Trump. These conspiracy theories involving the Ukraine are also part of Russian disinformation intended to re-elect Trump.

    It is time to hold Fox accountable for their participation in this conspiracy to place a Russian puppet in the presidency. This is an invasion of our country as surely as any military attack and we need to defend ourselves from it.

    Somerby is attacking the wrong journalists. It is the right who is complicit in this foreign attack on our democracy. By attacking the remaining legitimate news sources, Somerby gives aid to this foreign attack.

    This is serious and it is time we all took it seriously.

    ReplyDelete
    Replies
    1. The idea that the Internet Research Agency is a threat to anyone or anything is so laughably stupid that the person espousing such a notion should not only be ignored, but shot on the spot in order to remove his or her defective genes from the gene pool.

      Delete
    2. You mean the agency that Mueller investigated, prosecutors charged with criminal violations, and a grand jury indicted? Mueller, the prosecutors, the grand jury are all stupid?

      And, even better is your suggestion that they be shot. How very tolerant and democratic of you.

      Delete
  2. Somerby calls these "boutique" nations: Denmark, Norway, Belgium, The Netherlands, Austria, Switzerland

    That term is ridiculous. I'm sure these countries take themselves as seriously as the US does. Having large amounts of unoccupied territory doesn't make a country any more of a legitimate nation than these more densely populated areas of Europe.

    There was a time when The Netherlands was the center of world trade and Switzerland was the center of world banking. Belgium is the current head of the EU. Denigrating these countries with a word like "boutique" is ridiculous.

    ReplyDelete
    Replies
    1. Those nations' entire populations are less than the NYC or LA metro areas, moron.

      Delete
    2. 1:32
      Nice ad hominem there.

      Somerby specifically calls these nations “smaller”, meaning, presumably, population-wise. But then he adds the word “boutique.” What is “boutique” supposed to mean that isn’t conveyed by the word “smaller?”

      Delete
    3. How about that the population meets specialized criteria (like a specialty clothing store) as opposed to having diversity (like a department store)?

      Delete
    4. @deadrat
      Sure. If you think that’s what he means. In other words, “boutique nation” is a nation without racial minorities? No one has used that term this way, to my knowledge. But it seems to underscore Somerby’s desire to ignore black and Hispanic scores on the Pisa. But then, Japan, Taiwan, Australia: what levels of diversity do these larger, apparently non-boutique nations have? Couldn’t they be called boutique for the same reason?

      Delete
    5. Just a guess. That's why my comment has a question mark. George Will used the term 30 years ago to describe Kuwait as "a boutique nation, more of a country club than a country." And no, I'm not sure how that usage is apt here.

      TDH doesn't ignore black and Hispanic scores. He harps on the racial gaps. His point is that journalists like to spread the gloom about the state of US education, but the failures aren't systemic (as measured by Pisa) but affect only certain segments of our demographic. That's a problem to be faced, but not by deciding that the whole system is in failure.

      Delete
    6. There are two problems with your idea that the PISA scores show success for white students: (1) the scores have been stagnant since 2000, and (2) the improvement in US ranking is due to decreases by other nations not increases in US scores.

      Is stagnation a problem? Yes, because it occurs in the face of efforts to improve our schools. It implies that those changes and efforts to improve have been ineffective, at least with respect to this particular test (which measures math and science, not solely reading).

      Somerby acts as if he didn't know that the US habitually wishes to consider itself #1 in the world, not 9th. This is part of American exceptionalism. It is a kind of failure to be less than first in these rankings. Yes, this is a situation that has existed for a long time, but many Americans will find it surprising that we are not #1 in these test scores. The doom and gloom of this reporter's headline needs to be considered in light of that mistaken assumption by readers, one that Somerby entirely ignores. American complacency about its educational efforts is a problem, perhaps the one that lets charter school enthusiasts and billionaires play with school systems instead of letting educational experts decide what is good practice for educating children.

      No mention of DeVos. Why? She has had 3 years to wreck things and no one seems to care about that, least of all Somerby.

      Delete
  3. If you permit the USA to remove its lower scorers and just include the white children or just the Asian children, then you would need to permit the other countries to do the same and remove any low scoring minorities from their data. Would the USA remain the highest scorers then? We don't know because Somerby insists on comparing apples and oranges and doesn't make that adjustment for the comparable nations.

    Instead, he likes to pretend that there are no minorities in other countries.

    What if you removed all of the low scorers, regardless of racial or ethnic identification? Then our high scorers would be higher than those boutique nations for sure! Why didn't the NY Times and Wash Post report that! Those poor exhausted kids in South Korea -- who ALL go to school day and night!

    Somerby in his ignorance wants to discount the importance of motivation. And he claims to have been a teacher!

    ReplyDelete
    Replies
    1. Math scores are very carefully not being discussed by Somerby. He has effectively "disappeared" them.

      Delete
    2. Which is interesting, because he used to report almost exclusively on the math scores from the NAEP, because they tended to show improvements over time, as opposed to NAEP reading scores.

      Delete
  4. “American white kids, viewed as a group, would be the highest-scoring nation in the world!”

    And, in a similar fashion, if you removed the scores of blacks and Hispanics from the NAEP, the average score would go way up! Thus proving how amazingly well our schools are performing!

    Of course, this is called cherry picking, and it is par for the course here at TDH. When he wants to castigate the news media, he cherry picks from the test results to show how amazingly well we’re doing.

    But, last I checked, our school system consists of more than whites and Asians. One can only imagine the howling of the Howler if a news report had disappeared the scores of blacks and Hispanics in order to craft a positive story about schools.

    (For those who care, the numbers are:
    “Hispanic and Black students had lower average scores (481 and 448, respectively) than the U.S. average score.””).

    He also says thus:
    ‘In the Post's formulation, the new results from last year's tests "show widening disparities between high- and low-performing students in the United States, adding to a growing body of evidence showing worsening inequity in public schools."’

    Checking that handy-dandy link to the data Somerby provided, we can find this information:
    “The U.S. score gap between the top and the bottom ESCS quarters was lower than the score gaps in 2 education systems and higher than the score gaps in 34 education systems.” (Not good.)

    Of course, if you eliminate those bottom quarters, you see an amazingly positive result!

    Funny how that works.

    ReplyDelete
    Replies
    1. I think the point is that the generalized doom-and-gloom stories are wrong. US schools aren't failing across the board, and in pushing such a line, journalists fail to report on the areas in which our school systems don't work properly.

      Not so funny now, eh?

      Delete
    2. And yet, based on those pesky achievement gaps, you could say our schools are failing a significant portion of the students, and thus our school system as a whole isn’t truly working as it needs to. Isn’t that the point of worrying about the gaps?

      Delete
    3. If a significant portion of the system isn't working, but a significant portion is, then you could say that the system as a whole isn't working, but only if your brain isn't working.

      Delete
  5. “These are American white kids across the board, including kids in Appalachia and the Rust Belt whose families are devastated by Oxycontin. That average score even includes the fumbling efforts of white kids in Mississippi and Alabama, on whom we all know to look down.”

    One assumes the test administrators attempt to get a cross-section of students from different socioeconomic backgrounds and different geographical regions, but it’s hard to know precisely how many or if any students whose “families are devastated by OxyContin” actually took the test. It’s doubtful that “OxyContin use” is a data point collected by the test administrators.

    Somerby engages in a romanticization of the white kids taking the test, calling them “ragtag” and yet implying their pluck and resourcefulness, even in the face of liberal hatred.

    One wonders what you would see if you disaggregate the white score by socioeconomic status or geographical region. He was quick to ignore the results for blacks and Hispanics; here he is keen on looking at the aggregate.

    ReplyDelete
    Replies
    1. If you disaggregate the white score by income and region, you'd find some classes and areas that do even better than the aggregated white score. The point is that even with poor and benighted included, the white aggregate belies he overall gloomy stories.

      Delete
    2. In other words, when you cherry pick the data, things look less gloomy. Got it.

      Delete
    3. You're still confused. Got it.

      Do you see a difference between a system that works well for one segment of the population and not for another and a system that doesn't work at all? Which do you think our educational system is (as measured by Pisa)? Do you think it makes a difference when you try to fix the system?

      Delete
  6. What's particularly interesting is that, despite all the talk about our gaps, when you disaggregate you see that our black kids beat a whole bunch of nations as well.

    ReplyDelete
    Replies
    1. So, those horrible racial achievement gaps Somerby kept pointing out and moaning about, for years? Are they a problem, or not? Our blacks do better than other countries around the world! But those punishing gaps! But those Pisa scores! But those gaps...

      Delete
    2. The gaps mean that our schooling is failing certain segments of the population. That's important enough that journalists should stop writing misleading stories of the overall failure of our educational system.

      Get it yet?

      Delete
    3. “Failing certain segments”...you do realize, deadrat, that our public schools are approaching majority minority status, ie whites are becoming the minority, and in many urban areas, whites are almost nonexistent? I’d say that’s more than failing “certain segments”.

      Delete
    4. Many of our urban school systems are already majority minority. How does that affect my point? The National Center for Education Statistics says that there were about 50.8M students in public K-12 this fall. Here’s their demographic breakdown:

      23.7 million White students
      13.9 million Hispanic students
      7.7 million Black students
      2.7 million Asian students
      2.1 million students of Two or more races
      0.5 million American Indian/Alaska Native students
      0.2 million Pacific Islander students

      Can you spot the segments we’re failing (according to Pisa)? Can you spot the segments that we’re not?

      Or do you think the whole thing is a shambles?

      Delete
  7. This constant fiddling with statistics is tiresome. Looked at one way, things are terrible. Look at the data differently, and it looks great.

    It ultimately reduces education to a numbers game, where different sets of numbers can show different things and can be manipulated for ideological purposes.

    Yes, the media does this, but it reflects the obsession that policy-makers at all levels of government have with the numbers. It isn’t limited to liberals, either. Numbers are used to justify all kinds of interventions in schools, whether it is a decision to close a school, execute a state takeover of a “failing” school, or promote the proliferation of charter schools.

    There’s no need for Somerby to play the same game.

    ReplyDelete
    Replies
    1. Somerby can point out the (supposed) flaws in news reports without cherry picking the data to write some “feel-good” story about our schools himself. Or is that asking too much? There are obvious flaws in our system. Eliminating black and Hispanic scores to improve the apparent results of a test is dishonest.

      Delete
    2. I’m typing as slowly as I can in the hopes you can follow.

      This. Isn’t. A. Feel-good. Story.

      It’s a story of journalistic malfeasance in reporting the systemic failure of our educational system. But the system isn’t uniformly dire. According to Pisa it serves white and Asian students well. It fails black and Hispanic students. The latter, an obvious flaw. Do you suppose your plans to fix the flaw will change when you understand the difference between a partially-flawed system and a totally broken one?

      Delete
  8. Somerby keeps referring to the entities tested as “nations”, but the PISA test refers to them as “education systems.” They aren’t contiguous with “nations” in all cases.

    ReplyDelete
    Replies
    1. Contiguous means having a common border. You mean coterminous. In any case, yesterday TDH wrote

      Ignore the part about those four provinces [Beijing, Shanghai, Jiangsu and Zhejiang], which are famously unrepresentative of Chinese public education as a whole.

      So TDH did make the distinction clear.

      And you fail both the vocabulary and reading parts of this test.

      Delete
    2. “Yesterday?” It’s a good thing he didn’t say it five years ago, or you yourself might have forgotten. And as long as we get to ignore provinces because Somerby wants to, there’s no reason not to ignore black and Hispanic Pisa results.

      Delete
    3. What’s that mean? It’s a bad thing that TDH said it yesterday because you’ve already forgotten it?

      We get to ignore provinces in China when we measure our country against China.

      The reason to disaggregate US scores is not to ignore black and Hispanic students. It’s to point out how misleading it is for journalists to write their stock tales of a failed US educational system. The system works (on average) for white and Asian students (according to Pisa). Not so much for black and Hispanic students. Solutions for a totally-broken system will be different from solutions for a system that fails only parts of the population it serves.

      When journalists write misleading stories, people will be misled to the wrong solutions.

      What’s so hard about this?

      Delete
  9. "Even including all those kids, American white kids, viewed as a group, would be the highest-scoring nation in the world!"

    Are you feeling alright, dear Bob? What could possibly compel you to type this?

    ReplyDelete