Interlude—Andreas the Giant: Doggone it! The government shutdown has shut down the data we planned to use today as we discussed an important part of Amanda Ripley’s new book, The Smartest Kids in the World.
For more, see yesterday's post.
Ripley’s book is well written and very interesting. We also think it’s an amazing example of a type of new journalism, in which “Nordic robots” repeat preferred narratives which come from ranking elites.
Tomorrow, we’ll discuss a group of invisible children—a bunch of kids who get disappeared in Ripley’s book. But then, why shouldn’t those kids get disappeared? In the course of 230 pages, the TIMSS, the PIRLS and the NAEP all get disappeared too!
If we may borrow from our Shelley, only the PISA remains!
We don’t know when we’ve seen a book where so much data gets disappeared in service to preferred narrative. That doesn’t mean the book is all wrong. It means it’s a piece of work!
Jay Mathews called Ripley “a talented writer.” For today, you might scan her profile of Andreas Schleicher, the PISA’s inventor. The profile was published by the Atlantic in 2011.
Tomorrow, we’ll start with one part of that profile as we search for invisible kids.
What kinds of questions get asked on the PISA: Faithfully pushing a preferred narrative, Ripley says in her book that the PISA is “a smarter test” than other international tests, which she doesn't even name. On this basis, she simply ignores the data which come from the TIMSS and the PIRLS.
Is the PISA a smarter test? In 2009, Mathews wrote a blog post entitled, “Test that makes U.S. look bad may not be so good.” He started with a question from the PISA, a test which is designed to measure “creative thinking:”
MATHEWS (10/19/09): Politicians and pundits are using results from the Programme for International Student Assessment (PISA) tests to say our kids are falling behind the rest of the world, so maybe we should get some PISA practice. Brookings Institution scholar Tom Loveless, a member of the U.S. advisory board to PISA, offered this sample question for 15 year olds from the mathematics literacy section of the exam:That’s only one question from the PISA; there was more to Mathews’ post. Still: How sure do you feel that the PISA is “smarter” when you see a question like that?
For a rock concert a rectangular field of size 100 m by 50 m was reserved for the audience. The concert was completely sold out and the field was full with all the fans standing. Which one of the following is likely to be the best estimate of the total number of people attending the concert?
A. 2000
B. 5000
C. 20000
D. 50000
E. 100000
I think this is a bad question, and not just because I got it wrong. I said 5,000. The answer is 20,000. I don’t see why deciding four people, not one, would fit better in a square meter is a sign of math literacy. There are some people I don't like to get close to at concerts.
Loveless, an expert on international testing, agrees that the problem was ill-chosen. “I think it would throw kids off,” he said. “The math is rather trivial.”
Opinions differed in comments to Mathews’ post, which we found surprising. We’ll restate our basic stance:
If there are three major sources of data, it's silly to disregard two of those sources in service to preferred narrative. Presumably, the PISA data are worth reviewing. Presumably, so are the data from the TIMSS and the PIRLS.
That really wasn't a mathematical question.
ReplyDeleteEh.
DeleteLook: "Estimation" is in fact a subject in elementary/middle-school mathematics. In the USA, understanding metric measurements is, too.
The question is clearly related to those subjects. The question of whether 1 or 4 (or 0.4, 10 or 20!) persons per square meter is an appropriate estimate of the density of a field "full" of standing people is not an inherently un-mathematical one.
Also: "The math is rather trivial" is not a coherent criticism of the question, either. This is emphatically *not* simply a question on the subject of calculating the are of a rectangle!
While I would agree that the wording of the question could and should be improved, Jay Matthews' estimate that 1 person per square meter is "full" is at hilarious variance with most folk's conception of a "full" rock concert, I would think.
And for someone else, who is used to going to shows where you're packed in like sardines, and who knows that every promoter out there over-books their venues, it could be that 4 is too small!
DeleteAnon. @ 11:21 and Marcus hint at, but do not fully demonstrate the cultural bias of this question.
DeleteHow many Amercan feet can you put in a square meter?
In what countries would an astute pupil ask "what instruments can a rock play and why would any person in their right mind go to their concerts?"
It's a bad question. Period. From purely mathematical perspective, it's unsolvable, as not enough information is given. Grasp of the metric system does not lie in the domain of mathematics and neither does the knowledge of physiology, i.e how much space a person occupies. The proper answer to this question: unsolvable. There is something wrong with a person who would put such a question in a test.
DeleteAgreed. Horrible, awful question. How are these questions vetted before they get onto the test?? How many people preapproved that question?
DeleteOur "talented," privileged Amanda Ripley is a poser. She picks her "data" to support her contention that American public education is in "crisis" and needs a healthy dose of "reform." It just so happens that her "vision" of "reform" is consistent with that of charlatans like Wendy Kopp and Eric Hanushek.
ReplyDeleteThe late Gerald Bracey wrote that "comparing nations on average scores is a pretty silly idea. It’s like ranking runners based on average shoe size or evaluating the high school football team on the basis of how fast the average senior can run the 40-yard dash. Not much link to reality."
But the corporate "reformers" –– and conservative Republicans alike –– don't live in reality. Neither does the "talented," privileged Ripley. Ripley, who attended the $70,000-a-year Lawrenceville School & Country Club – replete with golf course, fitness centers and coaches, indoor skating rink, music and visual arts centers, and 12-person classes – seems to think that poverty and toxic stress have no bearing on school achievement.
The "talented," privileged Ripley thinks that too many people just make poverty a lame excuse for poor educational results.
Maybe she has a point. After all, she attended school in the lap of luxury. A school that allegedly nurtured, "character" and "scholarship," integrity, and "honesty," which is considered "crucial to moral growth."
So what's her excuse?
OMB (Questions asked, Some Answered)
ReplyDeleteWe asked some questions in our last comment on the most recent post. In the badly paraphrased lyric of a song from BOB's generation's formative years:
"Don't know much about Moral Equivalency. But I do know that BOB loves that Hypocrisy."
"The data in question would have been scores from the 2000 and 2009 PISA testing. We were going to show you how well different groups of American students scored on the tests." BOB yesterday
"If there are three major sources of data, it's silly to disregard two of those sources in service to preferred narrative. Presumably, the PISA data are worth reviewing. Presumably, so are the data from the TIMSS and the PIRLS." BOB, this post
Yes indeed BOB (Except when I add the tests you mention, there are four). As are data from nine years worth of tests, as you argued in one post, then conveniently left out a set from one year to make your argument in a later post because that year didn't just muddle your narrative, it destroyed it.
"Little bit crazy" followers of yours, like Teahadists and Moon Beamers you lament, seem to focus on how bad Ripley is, then ignore the fact you do the same thing while picking, chewing, repicking, rechewing, stewing, and puking your guts out over this lowest of low hanging fruit; popular "non-fiction" American commercial literature.
BOB says we need to focus on data from three tests. We'll buy that. What we won't buy is BOB's explanation for why Ripley ignored the other tests. Why? Well because BOB has been fudging figures on his own to serve his narrative, so why wouldn't he leave things Ripley may have said out as to why the other tests aren't that relevant to her analysis. We just don't know (we haven't read her book) and we don't trust BOB as a source.
TIMMS and PIRL measure age groups that are much younger than PISA. If we include them, can we, and should we compare results to see if there is a pattern of improvement within countries across the years as a given cohort of students passes its way through their own system? (O& can we Haz 1 examp. of silly ? frm each, pls???)
NAEP, which BOB often calls the GOLD STANDARD ( thus furthering, in his choice of figure of speech, the myth of the hard money monetarists)
cannot be compared internationally. It can be used to make some sort of judgement about internal American progress. But guess what? BOB, who deplores facts being DISAPPEARED, rarely looks at what happens when you compare 4th grade results to 8th grade results, almost always disappears the final ingot in his stockpile of GOLD testing, the "senior" NAEP. He rarely mentions it, and even less frequently gives his BS reason for ignoring it
When BOB does get back into his data base, I hope he focuses on the real DISPPEARED children he too often ignores; the American kids who are mostly, but not exclusively by any means, minority kids who drop out of school. The ones we don't test. The ones who, by some estimates, most of which are low ball estimates, comprise 25%. The ones who school districts around my state routinely underreport and totally lose track. You know, the ones who in all the other education data people like BOB, Ripley, and others pick over, manipulate, replicate, and obfuscate have truly DISAPPEARED.
KZ
Drop outs are not necessarily bad students who might score low on a test. In Hispanic communities, dropping out is a family decision usually related to a need for the child's labor, sometimes to work in a family business, sometimes to take care of younger children in the family, sometimes to bring in more income needed to keep the family afloat. When kids are leaving school for non-academic reasons, removing them from the pool is not likely to have much impact on the test means.. Another reason girls leave school is teen pregnancy. That does not happen for academic reasons, so removing those girls from the test pool isn't likely to affect means. Boys doing poorly academically are more likely to stay in school to participate in sports, so they would artificially lower test means offsetting the loss of males who leave because they are doing poorly in their classes. Bob has said repeatedly that the impact of dropouts on test scores is very complex. I don't think omitting them from a discussion is equivalent to entirely ignoring the impact of poverty on test scores, as Ripley has done. If you want to discuss drop outs, feel free. It would be better than the rambling garbage you have been posting here.
Delete"In Hispanic communities, dropping out is a family decision"
DeleteYou call my posts rambling garbage? Perhaps so.
The above sentence, however, is the best example of what I once heard an elected official describe as WHITE MISSIONARY LIBERAL IMPERIALIST THINKING.
If ever one needed evidence that Liberalworld is not simply a figment of BOB's imagination, but a real description of the state of the shared conciousness of BOB and his BOBfans, it is your entire reply.
KZ
http://www.youtube.com/watch?v=kKwXHDs1R6Y
DeleteThere's a good reason The Daily Howler ignores 112th grade NAEP scores. And that's because they should be ignored. They have little if any value.
DeleteJere Brophy and Carol Ames, two highly respected researchers, wrote this for the National Assessment Governing Board (NAEP's "daddy," so to speak) in 2005:
"Neither students nor school personnel are likely to perceive value in NAEP participation. Tests are not intrinsically interesting to most students, and although some may value rewards that successful test performance might bring, they usually do not value the process of test taking itself. This is especially true of high school seniors in their spring term, who are disengaging from the series of evaluation hurdles that is built into our high school culture."
Brophy and Ames added this: "NAEP does not align with their school’s curriculum and does not lead to feedback that would allow students to improve their school performance levels..."
And they concluded that "NAEP assessments, as conducted in the past, offers nothing of objective value to participating students, their schools, or their communities, so that twelfth-grade NAEP probably should be dropped."
By the way, the NAGB is one of those that maintains (falsely) that test scores are critical to the ability of the U.S. to "compete in a global economy."
However, the U.S. already is internationally competitive. The World Economic Forum provides an index of competitive nations.
When the U.S. dropped from 2nd to 4th place in 2010-11, four factors were cited by the WEF for the decline: (1) weak corporate auditing and reporting standards, (2) suspect corporate ethics, (3) big deficits (caused on by Wall Street’s financial shenanigans and implosion) and (4) unsustainable levels of debt.
Last year (2011-12), major factors cited by the WEF were a “business community” and business leaders who are “critical toward public and private institutions,” a lack of trust in politicians and the political process with a lack of transparency in policy-making, and “a lack of macroeconomic stability” caused by decades of fiscal deficits and debt that “are likely to weigh heavily on the country’s future growth.”
This year (2012-13) the U.S.dropped to 7th place. Problems cited were “increasing [income] inequality and youth unemployment” and “the United States is among the countries that have ratified the fewest environmental treaties.“ The WEF noted that in the U.S.,”the business community continues to be critical toward public and private institutions” and “trust in politicians is not strong.” Political dysfunction has led to “a lack of macroeconomic stability” that “continues to be the country’s greatest area of weakness.”
So, 12th grade NAEP is meaningless. And the NAGB is clueless.
One highly respected education researcher wrote this in response to your suggestion that the 12th grade NAEP
Deleteis meaningless:
"Neither students nor school personnel are likely to perceive value in participation in the 12th grade. It is intrinsically uninteresting for even those who return for the fall term. By spring what's left but the prom?"
I recommend we cancel the whole meaningless year!
Anybody who will amount to a hill of beans has already been recruited by the right college before they are finished being juniors."
KZ
If you could crowd 20,000 people into 5,000 square feet, it seems you could then crowd the world's population into Rhode Island and still have room for some chairs and cafe tables. Then you'd have the world's only Starbucks with a seat for everybody.
ReplyDeleteThat was easy. Tomorrow I'll figure out how to convert Connecticut into a parking lot.
You could work on creating world peace instead by turning Oklahoma into a Palestinian state. If the US government could unleash the Okies on the nations with whom we had binding treaties 100 + years ago, why can't we turn Sooner Nation into Sunni Nation now?
DeleteThen find a way to get the crackers out of the Florida Panhandle and make a real Jewish state we need fear nobody will bomb and nobody fears will bomb them.
Oh, and make Connecticut a car park instead of a parking lot. We need the green space.
I was proposing a mathematically based mental experiment, not a Final Solution. Jeez, everybody's so sensitive.
DeleteActually, I'm way ahead of you in our mutual pursuit of world peace. It's not necessary to turn over all 70,000 or so square miles of Oklahoma to the Palestinians. The Sunnis by and large are perfectly welcome. Hamas would have to come under probationary circumstances, like "one terrorist bombing and you're out of here." There's only 4 million of us Okies, so there's plenty of room.
But here's my idea from a long time time ago. In the days just before WWII, the Roosevelt Administration toyed with the idea of establishing a homeland for the Jews in the Alaskan panhandle. The idea never really went anywhere.
Well, I think that was the right idea, but wrong panhandle. Oklahoma's panhandle, basically an unsurveyed No Man's land until it was fused to OK territory in 1890, is roughly the size of Israel, especially when you add the two counties contiguous. It's a rather landlocked barren semi-arid region (not totally unlike the Middle East) used mostly for ranching, but it sits atop the vast Ogallala Aquifer, so water would be no problem. I'm sure the New Israelis would have no problem terraforming it into a New Eden in a couple of generations. The present population is only 35,000 and has been declining since the 1950s. They've always felt alienated from the rest of the state, and I'm sure with fair compensation they'd be willing to pull up stakes and fan out to states with which they feel a greater affinity.
All in all, it would make a great little isolated enclave the Jews of the world could feel safe and sovereign in, far away from Islamofascists and reasonably close to some great skiiing. And they finally might have some oil and gas to drill. And we Okies would no longer have to fly to New York to see some great musical comedy, financed by our winnings at various Indian casinos.
Er, I haven't gone and offended anybody, have I?
A final point, I wouldn't expect you to be too familiar with another state's history, and you were smart to point out that it was the US government that unleashed the Sooners in the Land Run of 1889. But the land involved was not Indian land at the time. Known as the Unassigned lands, they had been ceded back to the US by the Creek Nation under a new treaty.
All Indian Nations owned slaves and signed treaties with the CSA during the war and fought alongside the South. During Reconstruction new treaties were signed freeing slaves and giving lands in the western half of Oklahoma to other tribes.
If you really want to rise in defense of people who enslaved upwards of 10,000 native born Americans, I'd be glad to forward your post to Al Sharpton.
Once Obama's tar sands pipeline is built even the Chosen people will not be able to save the Ogallala.
DeleteI do appreciate the information about slavery and the First Nations. As a white person I feel less guilty about the smallpox blankets already.
At least the question is expressed in a context familiar to young people, instead of being about how many polar bears can fit on an ice floe.
ReplyDeleteIf you need a whole square meter for yourself, you'd better stay away from rock concerts. You should get about a square meter at a symphony concert, plus a seat (which I guess no rock concert has).
ReplyDelete"The answer is 20,000. I don’t see why deciding four people, not one, would fit better in a square meter is a sign of math literacy."
ReplyDeleteThe question is supposed to be about reasoning, not calculation. The possible answers include: 2000, 5000, 20,000, 50,000, 100,000. The lowest and highest are clearly wrong. If you measure someone's feet, they are about 12 inches for an adult male (worst case) and breadth of shoulders are about the same, 12 inches. So a person can fit in a 12x12 cube if necessary, but that wouldn't be comfortable, just possible. If you figure 9 feet in a square meter approx, then 9 people max = 45,000. That means the higher answer of 50,000 is not physically possible and 100,000 is likewise impossible. 4 people per square meter would be comfortable with room for some movement. One per square meter is further apart than even friends would stand at such an event and does not match the description of a "full" field so that is why 5000 is wrong. If 5000 is wrong, then so is 2000.
Kids should be able to reason this way. So should adults, but apparently not journalists. How can a journalist write an article about testing (which necessarily includes dealing with numbers) without being able to reason about numbers? The test makers made the alternative answer clearly wrong so there is no possibility someone doing the calculations would be tempted by the alternatives -- so I think this journalist just winged it and selected a round number that seemed like a reasonable concert attendance, instead of trying to figure out the answer. Estimation is not the same as a "close enough is good enough" approach to answering questions. I think that is part of the problem with the content of articles these days -- a sense that it doesn't matter if you get it right or not because who cares?
If half would agree to put their head up someone else's ass you could stack em two up in your 12 x 12 cubes. And if it was an Indigo Girls concert, smaller cubes could be used since most feet would not be worst case. So 100,000 is possible.
ReplyDeleteBased on a few minutes research, at least in the US, 7 sq ft. per person seems to be the "rule of thumb" lower limit for indoor occupancy, ignoring egress issues likely to increase the required space per person. So, you can do the calculations, but this completely rules out 20k or more people if this were indoors. So, 5000 would be the most reasonable answer if you take "sold out" to mean what it means when applied to a rock concert or pretty much anything else. The "correct" answer is clearly not. I've been to many rock concerts (in the US) of all sizes, indoors and out, and while people do tend to get a lot closer than 1 per square meter, there is always plenty of room in the back. Likely a well meaning question, but horribly implemented. If this question is indicative of the quality of the test, and I don't know if it is, it's a pretty crappy test.
ReplyDelete1 person, in a square meter? At a "rock concert"?
DeleteThat's your "most reasonable answer?"
You need to get out more.
I hope the people who worte this test are proud of what they have done to out once great nation.
ReplyDeleteDo you score questions ? It looks as though that might be appropriate. There was a time curriculum was set to cover a variety of topics and hopefully whet interest in learning. These days those setting the curriculum - whoever they might be - are more interested in how well they have bored subjects stiff and turned them off the experience of investigating the world. Learning tools were first priority - not tests.
ReplyDelete