DISAGGREGATION NATION: A very old regional tale!

WEDNESDAY, OCTOBER 3, 2012

Epilogue—Two last perspectives: If you want to compare the performance of the public schools of two different states, you have to “disaggregate” test scores.

If you don’t, you will end up thinking foolish things. For example, you will think that Maine has the best public schools in the nation. You will think that because the public schools of Maine have almost nothing but white kids.

You will even think that in a year when white kids in Texas are outscoring their Maine counterparts in Maine by large margins (see THE DAILY HOWLER, 9/27/12.) But then, you will end up believing many strange things if you don’t “disaggregate” test scores.

In her new book, As Texas Goes, Gail Collins doesn’t disaggregate scores, even though she explicitly swears by the practice. For that reason, she ends up insulting and mocking the Texas schools, even though the Texas schools are among our highest-performing.

How distorted is the portrait offered in Collins’ book? Today, we’ll quickly offer two more perspectives:

Score gains in Texas: For the last time, we’ll present the central passage where Collins portrays the performance of the current-day Texas schools. From this passage, a reader would think that Texas schools are low-performing and that they are failing to show further progress:
COLLINS (page 91): You may be wondering how things are going, education-wise, in the state that deeded its reform plan to the nation. Paul Sadler, the Democrat who led the effort in the Texas house, complains that the state is “testing our kids to death.” (Under the state’s newest regimen, students take seventeen high-stakes tests between third and eighth grade, and up to a dozen more while they’re in high school.) A survey by the Texas State Teachers Association showed that 43 percent of its members were seriously thinking of looking for another line of work.

Sadler still believes that the leaps made in the 1990s are holding up. “I think most of our schools do a pretty good job,” he said. Other observers are, at a minimum, disillusioned. “The eighth grade reading scores were exactly the same in 2009 as 1998,” said Diane Ravitch, referring to the scores Texas received in the national NAEP test results. (I know we were trying to avoid them, but sometimes it's unavoidable.) “The whole country is now embarked on remedies that didn’t do anything for Texas.”

David Grissmer, the author of that glowing RAND study, says that since 2000, when the study came out, Texas students’ scores on national tests have begun to “flag.” Perhaps coincidentally, that was exactly the time when Bush stopped being governor and turned the state over to Perry, whose interest in K-12 education was minimal. When the state’s budget developed a monster hole in 2011, Perry refused to raise taxes—or even dip into state savings—to avoid enormous cuts in school aid. As the impact began to hit districts, schools began cutting back on programs that had been in place since the Perot commission, seeking waivers on class size and preschool requirements. Former first lady Barbara Bush wrote an opinion piece in the Houston Chronicle protesting the lack of financial support for public schools. “We rank 36th in the nation in high school graduation rate,” she wrote. “An estimated 3.8 million Texans do not have a high school diploma. We rank 49th in verbal SAT scores, 47th in literacy and 46th in average SAT scores.”

It all sounded sort of familiar.
Earlier, Collins had quoted the late Molly Ivins describing the state of the Texas schools in the year 2000. According to Ivins, the Texas schools had “rocketed from abysmal to only slightly below average in a mere thirty years.”

A reader of Collins’ book would probably think that Texas schools were “below average” as of the year 2000—and that progress had frozen there. She would see data from Barbara Bush suggesting that the Texas schools ranked anywhere from 46th to 49th worst in the nation.

This reader would have no idea that Texas students rank near the top of the nation on the National Assessment of Educational Progress (NAEP), the federal testing program Collins (and everyone else) has praised for producing our most reliable data.

Are Texas schools showing progress on the NAEP? Given that statement by Ravitch and that phantom one-word “quotation” from Grissmer, that reader would have no idea that Texas students showed large score gains in math on the NAEP from 2000 through 2011.

Gaze on Texas kids' lack of progress! There were no NAEP math tests in 1998, the year from which the Ravitch comparison started:
Texas students, score gains from 2000 to 2011, NAEP
Grade 4 math:

White students, 11.86 points
Black students, 11.95 points
Hispanic students, 12.18 points

Grade 8 math:
White students, 17.11 points
Black students, 27.01 points
Hispanic students, 21.18 points
By a very rough rule of thumb, ten points on the NAEP scale is said to equal one academic year. Judged by that very rough measure, those are phenomenal score gains.

A reader of Collins’ book would have no idea that such score gains exist—or that all three groups of Texas students scored second or third in the nation on eighth-grade math in 2011. And by the way: Unless she’s completely uninformed, that statement by Ravitch should be regarded as flatly deceptive—as an act of intellectual fraud.

How did Ravitch come up with that vastly misleading claim—a claim which can be defended as technically accurate? First, she cherry-picked eighth-grade reading, the area in which score gains have been smallest all over the nation. Then, she failed to disaggregate scores, thus disguising the score gains which have occurred in Texas, even in eighth-grade reading.

(One further point: By failing to disaggreghate, Ravitch hid the large increase in minority and low-income kids in the Texas student population. This change in the state's student population helps explain the lack of score gains in its overall average score.)

Presumably, Collins had no idea how misleading Ravitch’s statement actually was. There is no sign that she ever examined any NAEP scores herself. And she had her educational research done by an architecture student!

This is the way our most famous “journalists” operate within our failed culture.

Texas versus New York: Collins typed her grossly misleading book in her lavish Manhattan penthouse, far from the “underclass” crowd. As she typed, she constructed an old and destructive, low-IQ novel, in which swells from the north mock southern rubes—the kinds of people who beat each other with frozen armadillos.

(In fairness, Collins weekends in Connecticut.)

What was the actual state of play as Collins constructed her novel? In her book and on her book tour, Collins frequently insulted the teachers and students of Texas. But here’s how the kids in her brilliant northern state compared to the kids in Texas on last year’s NAEP math tests.

We’re talking about the state of New York, not just New York City:
Students in Texas and New York: Rankings among the fifty states, 2011 NAEP
Grade 4 math:

White kids: Texas 6, New York 39
Black kids: Texas 4, New York 29
Hispanic kids: Texas 11, New York 39

Grade 8 math:
White kids: Texas 3, New York 30
Black kids: Texas 2, New York 18
Hispanic kids: Texas 2, New York 39
In eighth-grade math, Hispanic students in Texas ranked second among the fifty states. By way of contrast, Hispanic kids in the state of New York ranked 39th. And yes, these differences in national ranking correspond to large differences in average scores:

In eighth-grade math, Hispanic kids in Texas outscored their Empire State counterparts by more than twenty points! A similar 20-point gap exists between the average scores of the two states’ white students. (To construct your own data, click here, then click on “State Comparisons.”) That said, the gap between the two Hispanic populations is especially striking, since Collins has made many insulting remarks about the way Texas is allegedly failing to educate its Hispanic children.

Alas! As Collins wrote and uttered those insulting words, her home state was getting its keister kicked by Hispanic students in Texas! This was happening on the NAEP, the educational testing program Collins (correctly) recommends.

Can we talk? It would be hard to bungle a book more thoroughly than Collins has done. And yet, this book was written by one of our best-known national “journalists”—by the former editor of the New York Times editorial page!

When Collins appears on the Maddow show, she gets her ass kissed every time. (Every time Maddow does this, an angel loses his wings.) Routinely, “liberal” readers applaud her for the clowning she performs.

We’ve told you for more than fourteen years: American “journalistic” culture lies in an absolute shambles. This astonishing book by this badly failed person stands as a tribute to this much-denied fact. And yet, Collins’ book extends themes which can be seen all through our broken intellectual culture wherever public schools are discussed.

It's just as we have always told you: Everybody swears by the NAEP—but no one reports the NAEP’s data! (Except for the achievement gaps, which are used to make teachers look bad.)

All next week, we will be asking how this type of intellectual misconduct can happen. We’ll be asking how the press corps can tolerate this type of work. We’ll be asking why the liberal world tolerates this conduct.

Why do public figures in Texas accept this sort of thing? How about our “educational experts?” (In fairness, they are perhaps the most Potemkin of all our Potemkin elites.)

Unless you like pleasing regional novels, Collins’ book represents an act of intellectual fraud. How can it be that such work is accepted? (In many respects, this horrible book represents the long-standing norm.)

We’ll be asking those questions next week. Before we do, you can count on one thing:

The liberal world won’t comment or care. We quit on black kids a long time ago. Everyone knows and accepts this.

All next week: Why is this acceptable? Why is Collins praised?

Reading rankings for Texas and New York: Above, we showed you how students from Texas and New York ranked on last year’s NAEP math tests. We’re talking about the state of New York, not just New York City.

Here’s how they ranked in reading. Texas kids actually got outscored on one of these six measures:
Students in Texas and New York: Rankings among the fifty states, 2011 NAEP
Grade 4 reading:

White kids: Texas 10, New York 12
Black kids: Texas 7, New York 13
Hispanic kids: Texas 14, New York 19

Grade 8 reading:
White kids: New York 6, Texas 13
Black kids: Texas 10, New York 12
Hispanic kids: Texas 23, New York 33
New York students rank higher in reading than in math. We have no idea why that is—and there’s no sign that anyone cares.

10 comments:

  1. It's not surprising that Gail Collins would get her facts wrong. She's always been an airhead.

    It is surprising that Diane Ravitch would give a distorted analysis based on cherry-picked figures, because Ravitch has long been one of America's most respected experts on educational theory. Another question about Ravitch is why her POV on education switched from conservative to liberal. Perhaps these two questions are related, but any explanation I could offer would be a wild assed guess.

    ReplyDelete
    Replies
    1. Whatever may be the overall case with Ravitch, indicting her in such incendiary terms based on a single-sentence quotation in someone else's book is about as egregious an example of cherry-picked intellectual dishonesty as you can get. It all depends on context, and we are given no idea what the context of Ravitch's statement is.

      To the extent Somerby's context is critiquing the Collins' snide indictment of Texas education, he is absolutely correct. NAEP scores in Texas are generally very high when you disaggregate the data. That is important to know.

      In general, however, Ravitch's context is criticizing the validity of the claim started by George W that the kind of policies embodied in No Child Left Behind are responsible for the "Texas Miracle." Her statement about 8th Grade reading scores are essentially correct: white 8th grade student gains from 1998 are below the national average gains, black student gains are the same as the national average, and Hispanic student gains were far below the national average.

      Is it misleading to say that? Yes, 8th Grade disggregated math score gains for each ethnic group are substantially above the national average gain, but 4th grade math scores go in exactly the opposite direction: for each group in Texas, the gains are well below the national average gains. In reading, with the comparison date being 1998, 4th and 8th grade white and Hispanic student scores improved less than the national average; in reading, only black 4th graders improved faster than the national average, while black 8th graders' improved at the same rate as the national average. I'm not sure what the real agenda is here, but Somerby is being dishonest by not giving you the national gains for comparison.

      Of course, we do not see the full context in which Ravitch made her statement quoted by Collins, but the modest 8th grade reading gain below national average comes closer to meeting the norm of all comparisons than the math does.

      Those 8th grade match scores are the outlier, and yet NCLB-like policies are supposed to affect the entire educational package. There certainly is considerable reason to doubt that whatever changes W put in place had anything to do with improvements in NAEP test scores. It looks more like Texas figured a better way to teach math in later grades (but a worse way in early grades) than that getting tough had some magic effect.

      Molly Ivins' column in 2000 recognized huge gains in Texas scores, but contended, apparently correctly, that they were the result of changes that began in the 1960s, including better funding of schools with low incomes students, and long-pre-dated W's governorship. The data available for the early 1990s seems to confirm that: disaggregation by ethnic group shows that Texas students were already doing a lot better than the top-line aggregated score suggested. Ivins showed considerable pride in those gains. They were central to her column, and to suggest she was trying to attack Texas education by using two brief mentions of her understanding of where her state stood at the time -- not correct if you really understood disaggregation -- grossly distorts the essence of her column. She was not a policy wonk. Her subject was George W, and she was correctly ridiculing his claims to be running to be "the Education President" based on the alleged "Texas Miracle" that he probably had nothing to do with.

      It's a complex subject. Character attacks on people who are trying to make sense of it in the formation of policy have no place.

      Delete
  2. “The liberal world won’t comment or care. We quit on black kids a long time ago. Everyone knows and accepts this.” - lord robert de somerby

    >>> 1.) last poll I heard, blacks favor the gop pres. candidate by only 2% .
    2.) collins is trying to get obama elected
    3.) somerby seems to me be trying to regularly work the media refs to romneys favor.

    seems to me that as a practical matter, collins is more firmly ensconced in the black kids corner than somerby is.

    ReplyDelete
  3. "somerby seems to me be trying to regularly work the media refs to romneys favor because i am an idiot"

    Fixed that for you.

    ReplyDelete
  4. seems to me that as a practical matter, collins is more firmly ensconced in the black kids corner than somerby is.

    I think it's just the reverse. Because people like Collins promote wrong beliefs about what sorts of programs are effective for inner city students, the actual programs selected are more likely to be ineffective.

    This is not a minor matter. By twelfth grade, black students have fallen 4 years behind. Badly chosen educational approaches have done enormous damage.

    ReplyDelete
  5. Why do public figures in Texas accept this sort of thing? How about our “educational experts?” (In fairness, they are perhaps the most Potemkin of all our Potemkin elites.)

    Haha!

    ReplyDelete
  6. Despite the fact that Maine is mostly white it still has the highest poverty rate in New England. Because pverty in Texas might be more localized in black and Hispanic communities it is not hard to imagine a scenario in which very poor white kids in Maine outperform very poor white kids in Texas. In other words, race is not the only way to aggregate/disaggregate this information.

    ReplyDelete
    Replies
    1. "it is not hard to imagine"

      Yes, especially if you prefer uninformed speculation, as you seem to...

      Delete