That poll about Arab opinions on the US? Oops... Enron math...

Rm95:

Read my first post in this thread re: sampling.

Your example would boil down this way: there would have to be two separate polls, with separate results, both for the Utah and the Texas populations - because they are obviously two totally different populations. They are different populations because they were framed as different populations. Now, If you'd have framed it in terms of "Basketball fans", and simply polled 1000 people in Texas and 1000 people in Utah, then you'd have a single population, and aggregation would be possible.

What Gallup did here is in essence conduct 9 separate polls with 9 separate populations, but it was reported as a single poll with a single population, and that was errant. I don't know who screwed the reporting up, but I'm inclined to think it was CNN, as Gallup explicitly warned against aggregating, and CNN screws up and misrepresents everything else they get their hands on.

Your last comment - I personally think (and have thought) that much of the non-Arab Islamic world is less enthusiastic about wanting to see Americans die. Indonesia is not the problem, but Saudi, Kuwait, Jordan, etc certainly is.

 

No, the result for Indonesia is around 4%, and the result for Kuwait is around 36%. Where in the hell do you come up with 4.6% from that?

Because Indonesia has ONE HUNDRED TIMES AS MANY PEOPLE! A poll is meant to represent the total population. Sampling is used simply to avoid having to poll everyone. You can't just average the two numbers (what Gallup did) to come up with the aggregate. You have to take into account the population that each number represents. That's just basic statistical analysis.

as the sample was weighted in the first place

No, it wasn't -- at least not in the proper weightings. As mentioned in the original article, Kuwait was treated as an equal to Indonesia when totalling it up. That's the whole problem here. Each country was treated as being the same size when they aggregated it. There's nothing wrong with totalling the numbers ... But you can't add them up and divide by the number of countries. See my sample poll above to see exactly what Gallup did.

I am trying to tell you here that aggregation is not the way this should be done -

Sure it is. It just has to be done properly. If you're going to come up with a single number to say how the "Islamic World Views the US", then yes, you can aggregate it. Sure, there will be significant differences country-to-country, but there's nothing fundamentally wrong with aggregation. For more details, you can look at the country breakdowns.

And BTW, have you totally forgotten about the "Reporting" quote I posted? "Scientifically proper"... S*it, even Gallup is advising against it.

Gallup advises against doing it their way [with damn good reason, since they should never have even done it their way]. Why they didn't total it properly, though, I have no clue. There's nothing unsound about that, and the polling data they did compile is more than enough to do a proper aggregate.

The results are still valid on a country-by-country basis, no matter how you try to stack it up.

Absolutely. However, what the country-by-country basis shows is that a bunch of tiny countries dislike us. It doesn't show that a majority of Islamic people dislike us. The actual facts are, based on his poll, if you asked every Islamic person in all those countries, you'd get a small minority of them disliking the US. Those are the facts of the poll, and that was what was fundamentally misrepresented by Gallup and subsequently CNN and other media outlets who were given their news release.

 

By the way, while they "encourage readers to look at the poll on a country-by-country" basis, they don't provide those numbers publicly. You have to purchase the report for those details. Therefore, as far as the public is concerned, we can ONLY work with an aggregate number, which Gallup was kind enough to provide (to wow people and sell the paid report, of course).

Unfortunately, they f***ed up in providing it because they didn't do it properly. If they had reported the correct numbers that say 15% of the Islamic world hates the US, people would have said "oh well" and no one would have purchased their product. Not saying that was necessarily their reason, but it worked out pretty well for them...

 

Ahhh... So you are trying to proportionalize the populations? That is very, very bad statistical analysis, as any Stat 201 prof will tell you. Go back and review your sampling principles. A sample size of 1,200 or so will accurately guage *any* sized population within a margin of error of +/- 3-4 pts - and that is exactly the sample range used in this study.

Gallup gave some graphical "totals" of what it would look like if the results were aggregated, and then warned that the aggregate was just for fun. Someone took that and turned it into news, and that's the root of the mess.

And see my point above. Statistically speaking, because of the sample size - over 1,000 generally - the results for Kuwait and Indonesia were essentially perfectly weighted, with roughly equivalent margins of error. You don't need to poll more people in one country just to weight it better with a smaller country - it is the sample size that counts, not the size of the population. That is a *very basic* and well founded stastical principle, and it's what allows Gallup to consistently predict election outcomes for a country of 260 million people based on a sample of 1100 respondents.

I would very much like to see a poll of "the entire Islamic world", but that is totally impossible because there are a multitude of separate populations within that larger population, and there are barriers in many instances to conducting such a poll. This poll hit about 40% of the Islamic world, and considering the restraints (political, temporal, financial) this is probably the best one can hope for.

Just to illustrate this point, it would be easier to conduct a poll of the attitudes of "the Western world" - and that would be impossible as well for reasons that should be obvious.

Properly? You mean why didn't they proportionalize it? Statistics is not an egalitarian democracy where every voice gets a vote, man. See above on sampling.

Also remember - this is 9 separate polls in actuality, with 9 different populations. Impossible to aggregate accurately - again, whoever reported the "totals" f*ed up.

What it shows is that a bunch of Arab countries dislike us (as well as Pakistan - very important, as it is the second most populous Islamic nation). It also shows that Indonesia (and Morocco, not that it matters) are relatively indifferent, although there is still some animosity towards us. What CNN or whoever got wrong was the blanket statement of "the Islamic world hates us"; what they should have said was "the Arab world dislikes us, and a considerable portion of the Islamic world does as well".

That is the true result of the poll, and that's what everyone should stop trying to ignore. Thinking that the Arabs don't dislike us is a very dangerous delusion to hold onto.

 

Major: I'm glad you are the one arguing and not me.

treeman: I don't spit on anyone. Ever.

 

And see my point above. Statistically speaking, because of the sample size - over 1,000 generally - the results for Kuwait and Indonesia were essentially perfectly weighted, with roughly equivalent margins of error.

Dude, I'm sorry, but this is simply false. Yes, they were sampled properly. About 800 Kuwaitis were polled, and a little over 1000 Indonesians. However, the next step is incorrect. Summing up the total of the 1800 people as representing the people of Indonesia and Kuwait is completely and totally incorrect. You can ask ANY statistician and they will tell you unequivocally that this is the case.

What you have is that the 1000 Indonesians answered at about a 4% rate that they hate the US, and Kuwaitis answered at a 36% rate. Now, you CANNOT just add the 1800 people together and come up with a representative sampling. That's where Gallup f***ed up, even by their own admission.

The 4% and the 36% totals *are* statistically significant samples. However, to tabuate a total, you have to multiply each by the total population they represent and then divide by the sum-total population.

In other words, (4% x the population of Indonesia + 36% x the population of Kuwait) divided by the (population of Indonesia + Kuwait).

To put it in simpler terms, let's say you have two bags of balls:

Bag A has 10,000 balls (2000 red balls, 8000 blue balls)
Bag B has 100,000 balls (80,000 red balls, 20,000 blue balls).

You can do a statistical sample picking 1000 random balls from each bag. Results will be (close to):

Bag A: 200 Red, 800 Blue
Bag B: 800 Red, 200 Blue

You CANNOT total the 2000 balls sampled and decide that about half the balls are blue and half are red. This is what Gallup did.

You have to take the percentages of each population. Let's say we're totalling "Percentage Red":

You do (20% Red x 10,000 Total in Bag A + 80% Red x 100,000 total in bag B) / (110,000 total Balls). That is (2000 + 80000) / 110,000, or 74.5%, which is the exactly how many total Red balls there are in the two bags.

Similarly, when doing this poll, you have to take

(4% x 200,000,000 Indonesians + 36% x 2,000,000 Kuwaitis) / (202,000,000 total people) = 4.3%, or 8.7 million of the 202 million people.

I'm sorry -- I don't mean to be rude or condescending, but this is very, very basic statistical analysis. If you presented the Gallup Sum Total as a reasonable aggregate total, you would be laughed at by any statistics expert.

The poll numbers said, in no uncertain terms, that a minority of the Islamic world hates us, while it was verbally and statistically represented by Gallup's aggregates that a majority of the Islamic world hates us. That is completely false.

What CNN or whoever got wrong was the blanket statement of "the Islamic world hates us"; what they should have said was "the Arab world dislikes us, and a considerable portion of the Islamic world does as well".

Unless you have the individual country breakdowns, we have no idea if that is true or not based on this poll. If you do have those numbers, please share. Until then, the only thing that the poll showed us is that the Islamic world as a whole does NOT hate us.

 

For starters, the sampling threshold I mentioned is most certainly not false - I *dare* you to ask any statistician about it.

Second - where did anyone -except for yourself - add up the Indonesia and Kuwait totals and represent them as an "average of Indonesia and Kuwait"? You are the only person who has done that to my knowledge. What Gallup did was aggregate findings for all of the 9 nations - they didn't single out Kuwait and Indonesia, as you have - and develop a "total" (that I remind you again, they said was just for illustration - fun).

We don't know how they aggregated them either - you're just assuming that they aggregated the sample. Maybe they did, maybe they didn't; they could have aggregated results, and not the sample. You don't know either way, so stop acting as if you know their unreported methods.

Still irrelevant, though.

I do not know how many times I am going to have to say this but... Population size is totally, 100% irrelevant. Again, it is the sample size that matters. That is a *very basic* principle of statistical analysis that I again dare you to challenge any statistician about.

A sample size of 1200 out of a 10 billion large population is going to be far more accurate than a sample size of 500 out of a 5 million large population, even though it may not be obvious why. It has to do with a sort of statistical "law of diminishing returns" principle - the threshold has to do with the margin of error you're willing to accept, and at Gallup's typical margin (+/- 3-4%), that range is around 1100 or so. Again, I *dare* you to argue that principle with a statistician.

This is very basic statistics, and the fact that you can't understand that the sample size is of paramount importance leads me to believe that you've either never taken any stats courses, or you've forgotten everything you "learned" when you did.

BTW, if you tried to represent the "sum total" here as anything other than an exercise, then you should be laughed at. I suspect that's why Gallup gave the 'disclaimer' in the "Reporting" quote I gave. It is not supposed to be used in a serious manner, yet you and others are apparently fixated upon it at the expense of the real results.

As for the Islamic world as a whole not hating us - that is of course true. Hell, Turkey is pretty favorably towards us. But it is very safe to say that the Arabic (plus Pakistan, and Iran to a lesser extent) world generally dislikes us - that is virtually indisputable based upon the countries polled. If you continue to ignore that, then you are in denial, plain and simple.

I wish they would have surveyed Egypt.

 

Why doesn't population size matter?

If this is supposed to be a survey of the ENTIRE Muslim world, than the average results, as optained through a statistically significant sample size, in a country with 200 Million Muslims must be weigthed far greater than a country that has only one two million muslims, because these are two different populations being surveyed, as the differentiation in results shows. Otherwise the poll is ****ed up.

This seems to be Major's point.

Also, unless the report shows a result that suprises people, why would people buy the poll from Gallup? The true result of the poll doens't really suprise anyone, the "Just for Fun" numbers do. I may be no statistics expert, but 1 + 1 = 2.

 

I have a question for treeman since I am not incredibly knowlegable about statistics. I know just enough to know that statistics is at times incredibly counter-intuitive.

You say that 1200-1300 Muslims polled could be extapolated to a whole.

I have 2 situations here:
1200 Indonesians are polled, 4% hate the US.
1200 Kuwaities (sp?) are polled, 66% hate the US.

It seems (and please correct me if I am wrong) that you are saying you can average the two to say that 35% of all muslims hate the US since the total of 2400 polled could be extrapolated to any size population.

Now it also seems that the 1200 polled in each country could be exptrapolated to represent the entire muslim population. If that is true, why does the aggregate change so dramatically.

You say the following: "A sample size of 1200 out of a 10 billion large population is going to be far more accurate than a sample size of 500 out of a 5 million large population, even though it may not be obvious why. It has to do with a sort of statistical "law of diminishing returns" principle - the threshold has to do with the margin of error you're willing to accept, and at Gallup's typical margin (+/- 3-4%), that range is around 1100 or so. "

But in the above situation that is not true. In the above situation, when Gallup has 1200 samples and then addd another 1200 samples the result changes from 4% to 35%.

It just seems that something, and I'm not sure what, is out of whack here. I'm not sure whether it's your argument, my reasoning, or both...

 

For starters, the sampling threshold I mentioned is most certainly not false - I *dare* you to ask any statistician about it.

Who the hell is arguing that 1000 people isn't enough to survey? Sheesh, read the damn posts. What you CANNOT do, is then add those numbers up. If you sampled the ENTIRE Muslim world randomly, then you could do so. However, then your "random sample" would have about 100 times as many Indonesians as Kuwaitis. If you random sample in each country separately, you cannot just add the samples up.

If you find a single statistician that believes you can do otherwise, they need to fired on the spot and never allowed to teach stats again.

Second - where did anyone -except for yourself - add up the Indonesia and Kuwait totals and represent them as an "average of Indonesia and Kuwait"? You are the only person who has done that to my knowledge. What Gallup did was aggregate findings for all of the 9 nations - they didn't single out Kuwait and Indonesia, as you have - and develop a "total" (that I remind you again, they said was just for illustration - fun).

Gallup did the full 9 countries. I only did 2 because that's the only data I have. It's an illustration to demonstrate what they did and why it's wrong. I thought this was pretty basic. And, no, they did not say it was "just for fun" when their people announced the findings as "53% of Islamic people hate the US" both on TV and in their faxes that didn't contain that convenient disclaimer.

BTW, if you tried to represent the "sum total" here as anything other than an exercise, then you should be laughed at. I suspect that's why Gallup gave the 'disclaimer' in the "Reporting" quote I gave. It is not supposed to be used in a serious manner, yet you and others are apparently fixated upon it at the expense of the real results.

You know, you can keep repeating this over and over and over til your head falls off. The bottom line is that Gallup themselves ignored the disclaimer and that resulted in bad results being released.

We don't know how they aggregated them either - you're just assuming that they aggregated the sample. Maybe they did, maybe they didn't;

Yes, we do. They even said so themselves when they said that their totals line is unweighted. For anyone who understands stats, they made it very clear how they summed the thing up.

As for the Islamic world as a whole not hating us - that is of course true. Hell, Turkey is pretty favorably towards us. But it is very safe to say that the Arabic (plus Pakistan, and Iran to a lesser extent) world generally dislikes us - that is virtually indisputable based upon the countries polled. If you continue to ignore that, then you are in denial, plain and simple.


Again, prove it. Show me a single number in the poll results that show that the Arab world hates us. The only way you can do that is if you have the individual country results, and you don't. Therefore, the poll results that we have (which is simply the 53% number) tell us NOTHING. The smaller private numbers we got from the article haven posted (4% in Indonesia, 36% in Kuwait) show us why the 53% is wholly inaccurate.

This is simply ridiculous, so this is my last post on the matter. To sum up,

(1) Population size is more-or-less irrelevent in a single sampling. However, you cannot sum up separate population samples and create an accurate result. You must consider population size. When you do so, it is scientifically sound to aggregate multiple samples to come up with a comprehensive total.

(2) Gallup told us to evaluate individual countries separately, but did not provide any individual country #'s to evaluate.

(3) Gallup created an artificial and inaccurate number in their "for fun" total and then reported that as the overall finding of the poll. They f***ed up.

(4) Gallup's poll *was* meant to survey the Islamic world, and not the Arab world. That is demonstrated by both the title of the poll (Opinions of the US in the Islamic world, or whatever) and the fact that they included non-Arab countries. It's all supported by the fact that members of their organization presented said totals (53%) as the "opinions of the Islamic world". As such, it was also inaccurate in that it did not measure ALL Islamic countries. If I was taking a survey of the US, polling only people in Texas & Oklahoma will NOT give an accurate result. The totals they provided there for "% of people in the Islamic world that hate us" would therefore have been inaccurate even if they had aggregated it properly.

(5) The only number available to us is the 53% number, which has been shown to be completely inaccurate (through the article, we now also have the numbers for two of the nine individual countries: Indonesia and Kuwait). Anything you claim the poll tells us is either (a) based on this incorrect number or (b) not supported by the data. That is, unless you have the paid data. Any claims you make about the Arab World hating us may be true, but they are NOT demonstrated in any way by the poll results we have available.

 

Obviously Gallup is run by Jews...ONLY Jews would release errors like this to make Americans support Israel...

Treeman that should get you going...

 

Puedlfor:

Population size matters when you're dealing with a small population, but not when you're dealing with a large one. There's a term for it that I've forgotten, but the basic principle is that the smaller your population size, the more important each sample element gets, and therefore a given sample size would be weighted differently than that for a large one. When you're dealing with populations in the millions, it is totally irrelevant. Again, just ask your local stats prof if you don't believe me... They'll know what it's called too.

haven:

Haven, do you have a reading disability? I have already said that the aggregate "totals" are just an exercise and are not to be taken seriously. Oh, and Gallup said it too - I posted a quote of their "Reporting" section in this thread that said as much.

So do you have a reading disability, or are you just trying to pick a fight?

dylan:

No, that is not what I'm saying. In fact I have repeatedly said that if you do that it is just an exercise for fun. You can throw the numbers together any way you want, but some ways you throw them together will be meaningless. That's fine as long as you tell people that it is meaningless, as Gallup has done.

If they were not different populations, then you could do that, and what Gallup did would actually be meaningful. But they are really 9 separate populations, so what Gallup did was not statistically correct or meaningful. And again, that's fine as long as you tell everyone that.

The problem here appears to be that either 1) Gallup did not warn the reporters that the "totals" were not statistically valid, or 2) the reporters, in their zeal to break a controversial story, ignored that part. I honestly don't know which it is, but I'm inclined to blame the reporters, as they regularly screw up details in the press, and Gallup clearly warned on its website that the totals were not to be used for serious purposes. But I really don't know, since I haven't seen the original fax...

As for the sa,ple size question, ask your local stats prof. He/She'll verify that.

Major:

*Sigh*... Major, I am not arguing that you can - I have repeatedly said that because they are separate populations, any aggregate you do is not going to be statistically valid. I am not arguing otherwise.

Actually, had Gallup framed the poll population as "the muslim world, and gave every single muslim on the planet - regardless of their location or national affiliation - then it would not have mattered a single bit how many Kuwaitis and how many Indonesians got polled as long as the sample size was sufficient and everyone had an equal chance of being randomly polled. That would have been a true poll of the "Islamic world" that could be extrapolated to everyone.

Of course, they did not do that - for reasons which should be obvious (mainly political and financial reasons, but more importantly because there's no way in hell a Bedouin goatherder would have the same chance of being polled as a Saudi prince). Such a poll would be impossible to carry out, so they did the next best thing: they selected 9 separate countries, and conducted 9 separate polls. That's about all that resources allowed for, and all else considered, they did an excellent job overall.

Keep in mind that this was the largest and most complex poll ever carried out in the history of mankind. The fact that there was only one mix-up in reporting the results is truly amazing - just a little perspective on the whole thing.

As I said, I don't watch much TV, so you may be right about Gallup initially reporting incorrectly. Neither of us know what was in the original fax sent to reporters, either - you're going on one reporter's word. Funny thing how we haven't heard anything about this from anyone else?

You don't know for sure either way, and neither do I. Stop acting like you do.

At any rate, I'm operating upon information I have now, and Gallup quite clearly states that the "totals" are just for illustrative purposes, and warns against using them to draw any conclusions.

And this statement is based upon that fax that neither of us has seen, right? Are you psychic?

Why don't you sign up with Gallup and read the damn thing yourself? I'm not going to post it here because unlike the news snippets I'm always posting, it is not free information that can be passed around like so much gossip. We can talk about figures quoted in the press if you want, but...

Sign up with Gallup. It's a good service to have anyway.

In the meantime, http://www.cnn.com/2002/US/02/26/gallup.muslims/index.html .

Suffice it to say that the numbers from Kuwait, Saudi, and Jordan (as well as Pakistan and Iran) were very unfavorable towards us. Granted, that's not the entire Arab world (as I said earlier, I wish they would have polled Egypt in particular), but you can see a pattern here. It's a pattern I noticed years ago, which is why the results of the poll didn't surprise me.

I'm beginning to think that you just don't understand the relationship between population size and sample size. See my comments to Puedlfor, as I'm getting tired of repeating myself. Better yet, just ask a stat professor about it. Again, I *dare* you to do the latter. With large populations, the absolute number is irrelevant as long as the sample size is sufficient.

And I have already said - numerous times - that you cannot add up the different samples/populations. Why do you keep insinuating that I'm saying otherwise?

Why don't you by the damn report then? You know, no one paid Gallup to conduct this poll (to my knowledge), and they spend quite a bit of mula on it. There's a reason they want to sell it to you. I know, it's a bit expensive, but... If you're really, truly interested, that's the only way you're going to get it.

Maybe they did, maybe they didn't. I don't know which is true, but maybe they did screw up. It still doesn't invalidate the results of the poll - and that is my only real point here.

Gallup's poll was meant to survey as much of the Islamic/Arabic world as possible, given the financial, temporal, and political constraints. Obviously, they did not survey the entire Islamic world, nor the entire Arabic world. But you can look at the countries they did survey, and you can see a pattern. This pattern is what everyone here apparently wishes not to acknowledge... Denial is your loss.

I don't know what else to say except that you should buy the damn report if you really want to know the results. Gallup did not just give it away to everyone for a reason.

I am willing to bet that in the fax that was sent to reporters, Gallup gave up alot more than the "totals" - they probably sent out the entire results package. Blame the reporters for not giving you a $1250 report for free.

Buy the report. (I'm starting to sound like a broken record)

If you can read it and walk away still believing that we are loved by our enemies, then I bow down to your awesome powers of denial. You are a God.