Part 3—First approximations: Very large “achievement gaps” exist in American schools.
In yesterday’s main report, we looked at the range of scores in Grade 8 math on last year’s NAEP math test. We refer to the National Assessment of Educational Progress, the 43-year-old federal program which is widely regarded as out most reliable testing program.
Students took the NAEP last year. Based on conventional rules of thumb, a very wide range of math achievement is suggested by these scores:
Average scores, Grade 8 math, 2013 NAEPJust for the record, ten percent of tested students scored below that lowest score. Ten percent of tested students scored above 331.
90th percentile: 331
75th percentile: 310
50th percentile: 286
25th percentile: 261
10th percentile: 237
When conventional rules of thumb are applied, those scores suggest a very wide range of achievement levels among the nation’s eighth graders. As we all know, these gaps don’t appear completely at random among our students.
To a significant degree, these gaps in achievement correlate with family income and race. In recent weeks, major journalists seemed eager to underplay, avoid or deny that highly salient fact.
In a 10,000-word piece in The Atlantic, Nikole Hannah-Jones found a hundred ways to underplay the savagery of the achievement gap which faces low-income black kids. Meanwhile, in the New York Times and the Washington Post, Eduardo Porter and Professor Perry seemed to imagine a more pleasing world, a world in which black kids are “disproportionately” represented in public school gifted programs because they have been “channeled” or “funneled into a lower-quality education.”
Hannah-Jones made the same pleasing suggestion in her fascinating report about Tuscaloosa’s schools. For reasons we’ll detail in coming weeks, we think such journalism is a bit on the heinous side, though your results may differ.
Your results may differ! If so, we’ll suggest your results could be wrong.
For today, let’s start to define the size of the gaps by family income and also by race. How large are the achievement gaps based on income and race?
Again, we’ll use data from last year’s National Assessment of Educational Progress, the widely-praised federal testing program. Let’s start with family income.
In its public data, the NAEP employs a fairly crude measure of family income. It divides students into two large groups—those who qualify for free or reduced-price lunch, and those who don’t qualify.
This is not a measure of poverty, though journalists often present it as such. As a rough measure, a student qualifies for reduced price lunch if his family’s income is roughly twice the federal poverty level.
At present, this measure divides the nation’s student population roughly in half. According to NAEP data, exactly 50 percent of eighth-graders tested in math last year qualified for free or reduced price lunch. Fifty percent did not.
We’ll refer to these two groups as “lower-income” and “upper-income students.” Below, you see the average scores in Grade 8 math recorded by these two groups:
Average scores, Grade 8 mathA gap of 27.17 points separated the average scores of those two groups of kids. By a very rough rule of thumb which is often applied to NAEP scores, that represents an achievement gap approaching three academic years.
Public school students, 2013 NAEP
All students: 283.62
Higher-income students: 297.13
Lower-income students: 269.96
That is a very rough rule of thumb, as we constantly note. We’d love to see the nation’s “education reporters” interview NAEP officials and other experts, if any exist, to estimate the range of achievement suggested by those average scores.
That said, a second point is very much worth noting—those are the average scores for students in each income group. Roughly half the upper-income students scored somewhere above 297. Roughly half the lower-income group scored somewhere below 270.
Considered that way, those average scores suggest a fairly large “achievement gap” based on family income. We’ll discuss some of the ways such gaps arise in the weeks ahead.
That said, the gaps between our major “racial” and ethnic groups were somewhat larger than that. Using the language of the NAEP, these are the average scores recorded by the four major groups:
Average scores, Grade 8 mathThe gap between white kids and black kids was slightly larger than the gap between higher-income and lower-income students. Meanwhile, our Asian-American kids made everyone look bad.
Public school students, 2013 NAEP
All students: 283.62
Asian/Pacific Islander students: 305.92
White students: 293.19
Hispanic students: 271.02
Black students: 262.73
Tomorrow, we’ll break these scores down further. We’ll look at the average scores of lower- and higher-income kids within each of the four “racial”/ethnic groups.
Before the week is done, we’ll also take a quick look at Alabama’s statewide scores. Here’s why:
Hannah-Jones’ lengthy Atlantic piece dealt with Tuscaloosa’s public schools. In the main, she discussed Tuscaloosa’s high schools, past and present, dating back to the 1960s.
In its so-called Trial Urban District study, the NAEP produces scores for twenty-one cities around the nation; Tuscaloosa isn’t part of this program. But when we look at the size of the gaps for Alabama as a whole, we may start to gain a new perspective on some of Hannah-Jones’ suggestions and representations, past as well as present.
What’s happening in Tuscaloosa’s schools? The question is very important.
Those schools are full of deserving kids. They want to live good lives; they want to serve and achieve.
In many ways, we thought Hannah-Jones’ representations were pleasing for the adult crowd but perhaps a bit less than respectful of those children’s interests. Your results may differ, of course. But are you real sure that they should?
Tomorrow: The size of the gaps, presented in more detail
To access all NAEP data: To access the NAEP data cited above, click here, then click on MAIN NDE (Main NAEP Data Explorer).
Click again to agree to terms; from there, you’re on your own. You’d never know it from reading newspapers, but mountains of data exist.
This post suggest the possibility there are no experts who can explain what NAEP results may mean. We don't know.
ReplyDeleteOne expert might be Professor Loveless. While he does not address
the questions directly raised by the Howler here in this post, all Howler readers might find his simply explained background on the statistics
of NAEP scores easier to digest than the Howler.
He also gives a good example of how those statistics can be maniuplated by those with an agenda.
http://www.brookings.edu/blogs/brown-center-chalkboard/posts/2013/11/13-interpreting-naep-gains-loveless
Somerby does neither of the things addressed in this article -- he doesn't leave out Wash DC and he doesn't aggregate scores that have different scales. This source (which you also cited yesterday) is irrelevant to this discussion. Stats can always be misused by someone with an agenda (or someone clueless) -- that is nothing unique to NAEP or this discussion. If you are accusing Somerby of somehow manipulating scores, please state how you think that has occurred. The background Loveless provides is helpful, but it mainly addresses why test scores with different scales cannot be combined. Somerby is here addressing stratification, something Loveless does not talk about.
ReplyDeleteIf you are going to just google NAEP and post random URLs here, people will understand that this is just another form of trolling, especially when you post the same URL as if it were relevant to two different discussions, as you did yesterday and now today.
Somerby at no time states that there are no experts who can explain what NAEP results may mean. If you are suggesting that you are wrong.
I find it offensive to have to defend myself from a personal attack for linking to a person with fairly sound educational credentials. Such is the nature of blog commentary I suppose.
DeleteLet's go over your attack point by point in the order I made them.
You say: "Somerby at no time states that there are no experts who can explain what NAEP results may mean. If you are suggesting that you are wrong."
I said: "This post suggest the possibility there are no experts who can explain what NAEP results may mean. We don't know."
Somerby wrote: "We’d love to see the nation’s “education reporters” interview NAEP officials and other experts, if any exist...
I did not say Somerby stated there were no experts. I said he suggested the possibility there were none. I could have quoted him directly. I think I paraphrased him accurately. To quote him directly this time: "Your results may differ! If so, we’ll suggest your results could be wrong."
My first point in reference to Loveless was his, in my opinion, very good and cogent explanation of the statistical variances in reporting NAEP results. I made no effort to criticize Somerby other than to say Loveless might explain it better than it has been explained here in the Howler. Again, your results may differ. Here I can't stipulate who is right.
I made no suggestion that Somerby did any of the things Loveless described the Govedrnor of Tennessee doing. The fact that you opened your attack on me with that accusation is interesting and points to someone acutely defensive about things imagined but not present.
Some of us are defensive because of the incessant trolling, which lately has taken to regurgitating anything they find on NAEP using Google, as if Somerby were somehow wrong because someone else has written a post about NAEP. KZ, for example, has been doing this. When you examine their links, they are always specious. Nitpicking Somerby's use of language -- he may have suggested there are no experts -- is another sign of trollery. If you do not wish to be mistaken for a troll, don't behave like one.
DeleteI see the point of greatest interest to you in my response was the part about you, not the inaccuracies in the comment you made to which I responded.
DeleteI have been looking for specious links by KZ and others.
I can't find any, but I confess I only went back a half dozen or so posts in this series.
Poor KZ. He doesn't even comment, and still people respond.
DeleteThe Howler repeats the same statistics presented yesterday on the range of scores on 8th grade Math results from 2013. It does show a big gap.
ReplyDeleteAnother blogger with a background similar to Somerby's has done this kind of analysis. Howler readers might find it useful.
http://gfbrandenburg.wordpress.com/2014/05/08/just-how-flat-are-those-12th-grade-naep-scores/
The main connection between these two articles is that they both discuss NAEP scores. Aside from that they are focusing on different points. If you see some other connection, please explain.
DeleteOne uses 8th grade math. The other 12th grade reading. Both address scores results based on the percentile of scores. Both show gaps. While Somerby does not directly address score gains today, that has been his primary focus in the series to date. Brandenburg has a different perspective on gains.
DeleteJust for the record both are former teachers who blog about education issues.
I like Brandenburg but its not appropriate to use 12 grade scores to compare across time.
DeleteEvery year, fewer and few kids drop out of high school. Academically challenged kids who would have dropped out are now staying in school and making up part of the NAEP sample.
As a result, you cannot accurately compare 12 grade NAEP scores year over year. My guess is that if 12th grade scores have remained flat then there has actually been at least a slight improvement over time.
I might like you if I knew you 1:05, but it is not appropriate to use drop outs as an excuse, as they commonly are, for ignoring and/or dismissing 12th grade NAEP scores.
DeleteFirst, there is no accurate measure of "droputs" even now. It varies from state to state. The only long term measure of high school completion rates comes from the Census Bureau, and it's common unit of measure used to measure the status drop out rate is "adults 16-24." Its count excludes as drop outs people who did drop out but got a high school equivalency certification. It does include people who never went to school in the United States. As a result, the drop out rate among Hispanics, which is the primary force in the dramatic lowering of measured long term dropouts, is subject to wide flucuations which will never impact high school senior test takers.
Second, the flat results from NAEP at both the 12th grade (Main NAEP) and 17 year old (Long Term NAEP) are as consistent among groups (high scorers, whites, the above average income) with less likelihood of dropping out than those groups whose retention may be reflected in any statistics used to measure dropouts.
Your guess may prove to be right. My guess is it won't. But I am not guessing when I say the measurement of dropout rates is so faulty that using it as an excuse for ignoring test scores among older students is truly just that, guesswork.
Somerby has been complaining because journalists, even those writing on education topics, do not explain key things (like the improvement across the board in academic performance, or the racial gap that persists despite that improvement). He is not suggesting there are no experts who can explain NAEP. He does complain that the experts (including education professors, presumably) have chosen not to get involved in explaining these things to the public. Loveless seems to be an exception, but he is not a journalist and he is not writing in a major newspaper but rather is focused on a specific local issue in TN. I don't see him as a contradiction to Somerby's thesis.
ReplyDeleteI suppose it is helpful to post other sources about NAEP, but really, anyone who wants to find more articles on NAEP can use Google.
"I suppose it is helpful to post other sources about NAEP, but really, anyone who wants to find more articles on NAEP can use Google." Anonymous @ 11:23
Delete"If you are going to just google NAEP and post random URLs here, people will understand that this is just another form of trolling...." Anonymous @ 10:59
Different keystrokes for different folks.
I see no contradiction. Both are saying it is unhelpful to post random NAEP articles in this discussion because (1) anyone who wants to can find that stuff themselves, and (2) it distracts from coherent discussion of what Somerby said.
DeleteNitpicking stuff like this is definitely trollery, so if you were to be given any benefit of any doubt, you have resolved that issue.
"(2) it distracts from coherent discussion of what Somerby said."
DeleteCare to point to any?
Update
Delete""(2) it distracts from coherent discussion of what Somerby said."
Care to point to any?"
Didn't think so. I will.
Urban Legend @ 3:10
Anonymous @ 10:44
Anonymous @ 11:12 and subsequent responses
Anonymous @ 11:23
David in Cal @ 12:26 but no responses
Confirmed troll.
DeleteCoherent commenter.
DeleteAs if
DeleteYes, academic achievement correlates with race and family income. So, what? Each student is an individual. An Asian student in the 10th percentile is just as much an outlier as a black student in the 10th percentile. The school faces the same challenge of how to educate either of them.
ReplyDeleteThat's right, David. Bob sometimes refers to our benighted ancestors, and their suppression of black literacy. But he's full of baloney. Slavery cannot possibly have any residual effects today. There is no need to address the challenges faced by individuals who happen to be black. There has been no systematic damage to the "black community."
DeleteYes, if only they'd been left to flower in the full richness of their intellect, as their ancestors back in Africa have manifestly done.
DeleteDavid, how can you call for treating kids as individuals and then suggest one-size-fits-all education?
DeleteIf I suggested one size fits all education, I didn't mean to do so.
DeleteFirst you argue that sources of difference don't matter. Then you say: "The school faces the same challenge of how to educate either of them."
Delete1:16 - apples and oranges, douchebag.
Delete3:15
Deletehttp://www.thefrisky.com/2011-01-06/mind-of-man-why-douchebags-are-called-douchebags/
3:15: actually, no.
DeleteAnonymousJune 4, 2014 at 1:01 PM -- you sound more interested in assigning blame or making excuses than in actually teaching black kids.
ReplyDeleteSuppose a particular black student is below grade level. Maybe he would have been a better student, but for the effects of slavery. I don't know how you can prove that. I don't even what that statement means, when applied to an individual student. But, let's assume that you're correct: slavery is somehow responsible for this child being below grade level.
I still say, so what? The school has a pupil who needs instruction that will be effective for him. I don't see how postulating the impact of slavery helps the school design appropriate books, teaching methods, etc. that will work for a student at this particular grade level.
David, don't you think a child with a cognitive, perceptual or auditory learning disability might need different types of materials than a child from a low literacy family background (with less early exposure to language)?
DeleteDavid has a severely constricted understanding of history. Nobody is limiting their observations to the era of legal slavery. There was also over 100 years of Jim Crow in the South and de facto segregation in the North. De facto segregation through a variety of techniques, most prominently housing, continues to this day. If you don't understand the history, your solutions are unlikely to work very well.
DeleteIf you don't understand the history, your solutions are unlikely to work very well.
DeleteI don't see that. Suppose you have a black student who has trouble reading. You need to find a way to motivate him and to teach him. You need appropriate materials and lesson plans. Maybe you want to choose readings that relate to his life and his world. You might want to schedule extra time. Maybe some sort of rewards would be helpful.
Now, suppose you also know that his grandparents lived under Jim Crow and his great great great grandparents were slaves. What would you do differently?
I might substitute something by Zora Neale Hurston for Dick and Jane Spend the Afternoon at the Yacht Club.
DeleteSeems reasonable. As I said, "Maybe you want to choose readings that relate to his life and his world."
DeleteHowever, I am not familiar with studies showing that kids learn to read better when reading stuff written by someone of their own ethnic group. I and the rest of my class in the Bronx did learn to read from Dick and Jane, even though their lives had little in common with ours. I also recall early days reading adventure stories that had little in common with me or my family.
I think it's more important that reading be taught in a way that's effective. E.g., the famous book "Why Johnny Can't Read" by Rudolf Flesch claimed to show that the whole word method wasn't a good way to teach reading. Also very important is that each student have reading material that's right for his level.
TDH says: "Average scores, Grade 8 math, 2013 NAEP
ReplyDelete90th percentile: 331
75th percentile: 310
50th percentile: 286
25th percentile: 261
10th percentile: 237
Just for the record, ten percent of tested students scored below that lowest score. Ten percent of tested students scored above 331."
It's not only Big Media journalists who are challenged by statistical concepts. If those are "average scores" for students in that percentile segment, then it does not mean everyone in that segment scored at or above the score. If those are scores that define where the percentile breaks, like colleges reporting their 75th and 25th percentile scores in official college data collection, then those are not average scores for the segment. This should be clarified, since both approaches are used for different purposes.
Why does everyone have to nitpick? I feel sorry for Mr. Somerby. He was even forced to do another post on D'Leisha Dent
DeleteWell, yes; it was necessary to do an update on the future of Ms Dent because he hadn't yet acknowledged the fact that she was accepted at a four-year college. That information was previously made available by some of the people who commented on his articles.
DeleteThere are some very, very smart kids in our schools. There are, sadly. some not so smart. We don't go to school with the kids we want. We go to school with the kids we have.
ReplyDeleteIt is good to keep this in perspective based on all the propaganda we hear. Tests can never tell the whole story any more than our brutal history can teach us even if repeated.
I imagine you'll mention this at some point, but not only is the white-black gap larger than the "rich-poor" gap (more on that in a minute), poor whites outscore non-poor blacks. Which pretty much kills any possibility of the gap being entirely about income.
ReplyDeletehttps://educationrealist.wordpress.com/2013/04/05/philip-dick-preschool-and-schrodingers-cat/
I did some work on the NAEP TUDA scores that suggest the gap between black students in, say, Boston as opposed to Detroit is linked to the different poverty levels.
https://educationrealist.wordpress.com/2013/12/31/naep-tuda-does-black-poverty-matter/
Brilliant series of posts, Bob. I am reading each thoroughly and taking notes on them, also sending them along to friends.
ReplyDeleteLTR
Be sure and send the post as well.
Delete"our Asian-American kids made everyone look bad"
ReplyDeleteIf these are truly acheivemebt tests they didn't make anyone look like anything. All they showed is what they can compartively acheive. By Asian American standards, not much,
Obviously, slavery has nothing to do with the poor academic performance of black kids today. Obviously, Spanish colonialism has nothing to do with the poor academic performance of Latino kids today. Yet those two groups continue to under-perform, on average. What to do? Impose standards they can't meet? Design curricula for them? Get them into Headstart and kindergarten? Spend other people's money on them? Or just eff them?
ReplyDeleteThe answer may lie in the mountains of data the newspapers don't tell us about. Or it may not be there at all. I myself am concerned about the range of averages in underperforming white children. It begs the question; do we have enough home grown Asiatics to be competitive in the next decades?
DeleteI know you are just trolling, but data show that Asian American students regress to the mean for white students with acculturation (the further they get from new immigrant status).
DeleteIt is more expensive to ignore poorly performing kids than to try to help them. When you ignore them, you will wind up spending more on them in the long run, through the criminal justice system, aid programs, lower tax revenues because they will earn less money, lost productivity and so on. Ignoring kids hurts our economy and our society (because we are all in this together).
DeleteSo I suggest we figure out how to help them. I vote yes on Headstart and kindergarten and research to design better curricula and teaching methods, and media program to educate parents and programs to include parents more in the mainstream of life (so their kids will benefit) and efforts to address inclusivity and lessen income inequality, and so on. In other words, all those progressive programs.
Other people's money? That is sort of like talking about your rain and other people's rain. Have you ever noticed how wrinkled a dollar bill is when you get it? That is because of all the other hands it has passed through. Our economy and our society both work like that.
That said, a second point is very much worth noting—those are the average scores for students in each income group. Roughly half the upper-income students scored somewhere above 297. Roughly half the lower-income group scored somewhere below 270.
ReplyDeleteWhat is an Average?
Is the post saying that these scores are the aritmetic mean and the assumption of a relatively normal distribtion is either being made?