That poll about Arab opinions on the US? Oops... Enron math...

http://www.sltrib.com/2002/Mar/03242002/nation_w/722197.htm
Islamic Poll Distorted by Averaging
Sunday, March 24, 2002


BY RICHARD MORIN
and CLAUDIA DEANE
THE WASHINGTON POST

The hugely influential Gallup Poll of Islamic Countries released last month continues to make news -- not all of it good for the Gallup Organization, USA Today and CNN.
The National Council on Public Polls, a leading professional watchdog organization, recently posted on its Web site a stinging rebuke of CNN and USA Today for the way their reporters reported the overall results of the nine-nation survey project.
But now it appears that Gallup itself provided reporters with the sensational characterizations that were the primary target of NCPP criticism.
It is also apparent that the hype was unnecessary, tainting an otherwise remarkable and important polling project, survey experts said.
One thing is certain: The polls made big news around the world. President Bush publicly expressed dismay about the results. So did a small army of diplomats, policymakers and pundits.
And little wonder. As reported by CNN and USA Today, the poll found that 53 percent of those interviewed said they had an unfavorable view of the United States. Only nine percent thought the U.S. military action in Afghanistan was justified. Fewer than one in five -- 18 percent -- in six of the nine countries believed Arabs carried out the Sept. 11 terrorist attacks. (Three countries wouldn't let Gallup polltakers ask the question.)
You won't find those results on Gallup's Web site, however. That's because of one rather big problem: These eye-opening results were "actually the average for the countries surveyed regardless of the size of their populations," the NCPP noted. "Kuwait, with less than 2 million Muslims, was treated the same as Indonesia, which has over 200 million Muslims."
That's Enron arithmetic. It's as if California and South Dakota each were granted the same number of electoral votes in presidential elections.
The problem would be particularly worrisome if there were big differences in the results across the nine countries. And there were, at least on some key questions. For example, 36 percent of those interviewed in tiny Kuwait said the Sept. 11 terrorist attacks were morally justifiable, compared to only 4 percent in Indonesia.
If the results of the two countries were averaged together, which is what NCPP said Gallup did, the result suggests that about 20 percent of these Muslims seemed to view the attack as justified.
But if the results were properly adjusted to account for population, the result is very different: About 5 percent of all Muslims in Kuwait and Indonesia thought the terrorist attacks were morally justified.
So where did reporters get the bogus aggregate numbers?
From the Gallup Organization, said Andrea Stone, the reporter who wrote USA Today's page 1 story. The aggregates were clearly noted in the first fax sent by Gallup to USA Today and CNN, which also included the country-by-country results. "I didn't do the arithmetic," Stone said.
Frank Newport, editor-in-chief of the Gallup Poll, acknowledged that the aggregates were included in the material supplied to news organizations. So that means Gallup itself is partially to blame for the snafu, right?
"I don't know," he said.
Newport said Gallup analysts repeatedly cautioned against using the aggregated numbers when the results were released at a seminar in Washington. But no such warning appeared on Gallup's first release to Stone or to reporters at CNN. And it was this release that reporters used to prepare their stories.
Gallup conspicuously omits references to the aggregate numbers in its own analyses of the poll data. Instead, it cites results from individual countries and strongly urges others to do the same. "Look at the (results from) individual countries, look at the differences in attitudes between countries," Newport said. "That's how we have chosen to look at it."
Newport is the vice president of NCPP, whose members include many of the nation's most respected and influential pollsters. He declined to say whether he endorsed NCPP's statement. Gallup is partners with CNN and USA Today on surveys in the United States, but the news organizations were not partners in the Islamic poll.
In its statement, the NCPP review board also faulted CNN and USA Today for repeatedly claiming the poll was a survey of "the Muslim world."
Turns out it's a small world -- much too small. The nine countries in which the survey was conducted are home to only about 40 percent of all Muslims, the NCPP noted. Included in the study were Indonesia, Iran, Jordan, Kuwait, Lebanon, Morocco, Pakistan, Saudi Arabia and Turkey. Conspicuously absent were views of Muslims living in India, Egypt, Bangladesh and Nigeria.

 

Perhaps, next time, you should contribute your opinion in a thread where we're actually debating content? In case you haven't noticed, we've had enough vitriolic clashes on this topic. I thought it was important to note that a poll that had made a huge splash on this board turned out to be ridiculously flawed (well, the interpretation of the data, at least).

If you really want to voice your opinion, perhaps you should resurrect a thread in which we actually debate the issue.

 

Azadre:

It's ok . Didn't mean to come off harsh... we've just had a lot of really heated discussion on this, already, and I wanted to avoid another brawl .

I agree that it's tough to survey "the Islamic world" when one only includes 40% of all Muslims.