« previous | TPM CAFÉ READER POSTS HOME | next »
Polling.. are there any poll buffs out there who can illuminate?
How are to we treat/explain the extraordinary differences in different groups of polls?
Times/Bloomberg yesterday had Obama leading 49-37
Newsweek on the 20th had him leading 51-37.
Gallup today has them tied. Rasmussen has them 49-45 if you include leaners. And at no stage over the last week have either of them come up with anything approaching the Bloomberg/Newsweek polls.
Why the amazing discrepancies? What are we to deduce from them? Is there any analytical/statistical reason that makes sense of this?
Advertisement











Comments (12)
For a good online place to read/discuss/understand polls, try Pollster.com a place for serious statistically-minded "geeks" (and I say that with respect and affection, being "poll-challenged" myself.)
June 25, 2008 9:44 PM | Reply | Permalink
Arrrrgh! damn links...
Pollster.com
and in the HIGHLY LIKELY EVENT the link fails:
http://www.pollster.com/
reezenfrazzithotdoggoneitdickdastardly!
June 25, 2008 9:50 PM | Reply | Permalink
UNCLE!...
Pollster dot com
I am going to go slit my wrists now with a very dull razor.
Goodbye cruel world....
June 25, 2008 9:55 PM | Reply | Permalink
LOL! Thanks, Jade. I did go there (and fivethirtyeight) before posting this question. Neither of them illuminated.
sigh
June 25, 2008 10:14 PM | Reply | Permalink
You might also try http://www.fivethiryeight.com
Excellent polling site
June 25, 2008 9:58 PM | Reply | Permalink
First, it helps to get a better picture of the polls by seeing more of them...
http://www.pollster.com/08-US-Pres-GE-MvO.php
Second, sample size and population are relevant. Pollster lists three types of population, RV, LV and A.
RV is registered voters, presumably who said yes to a question "Are you registered to vote?"
LV are "likely voters" most likely, people who answered yes to a question such as, "Did you vote in the last election?"
A is (my guess, I didn't read the whole site) all respondents.
Large sample size LV is the likely best survey. There are other characteristics that may not be reported such as whether the samples are clustered, the number of call backs (likely not many), the non-response rate, and for some, the use of internet and/or volunteer sampling (volunteer sampling is essentially invalid).
You will notice that there has not been that much change in the spread between the two candidates in the large sample size LV surveys over the past 20+ days (it ranges form 4 to 7 points).
The whole point of sampling is to REDUCE risk of error, not eliminate it. When you hear that the margin of error is +/- 4 points, the missing words are "with a x% confidence level." I don't know the industry standard for this purpose, but the universal standard is 5% unless the issue is "life or death." Five percent means one will be wrong, SUBSTANTIALLY WRONG, once every 20 times.
With an average of more than one poll a day, that would mean a substantial error (even without considering the RV, A issue) more often than once every 20 days. Over a longer time period, one would expect those errors to sometimes spread out and other times cluster.
June 25, 2008 10:03 PM | Reply | Permalink
um...
(pollstertwit here)
I did follow the link and am none the wiser.
Do you have a sense from all this which ones are the more credible? ie the dailies or those others which give him a big lead?
June 25, 2008 10:17 PM | Reply | Permalink
Yes, pollstertwit... READ WHAT I SAID ABOVE http://tpmcafe.talkingpointsmemo.com/talk/2008/06/polling-are-there-any-poll-buf.php#comment-2929433 . It lays it out for you.
June 25, 2008 10:26 PM | Reply | Permalink
I'm not a polling buff, but I read an explanation today at The Carpetbagger Report: http://www.thecarpetbaggerreport.com/archives/15988.html#more-15988
I didn't really understand it, but maybe it will make sense to you.
June 25, 2008 11:37 PM | Reply | Permalink
Ah, I see what is wrong with my answer. I didn't say the big spread is likely right. Well, my gut says it is, the math says it isn't. The one thing that is brought up at The Carpetbagger Report that I missed is the reweighting issue. If we think the shares of Dems and Reps are pretty much fixed, then reweighting is important, especially when polls don't do callbacks (I think). If we think that those shares are in flux, then reweighting can make the results unreliable. Too bad we don't know the answer.
June 25, 2008 11:57 PM | Reply | Permalink
Fran, my guess is that there are two explanations. The first is statistical and the other is methodological.
If five firms do a poll using the same methodology but asking different sets of randomly chosen respondents, they will come up with different numbers. Although the differences should be relatively small, within five percentage points or so.
The larger differences will be methodological. First, how the samples were selected and more importantly, how they were interpreted.
The way undecided voters are handled is clearly going to affect the final results.
In an "odd" year like 2008, things get complicated as it is harder for the pollsters to predict voting patterns. They do try to predict them, but different firms will have different guesses. Even with the same set of raw data, the interpretation may well be different.
In the US electoral system, it's all of course doubly complicated because of the Electoral College, where the number of votes is not as important as how they are distributed. National polls might even be misleading.
The bottom line is that we won't find until November who gets to be the next president :)
June 26, 2008 5:41 AM | Reply | Permalink
Was I whistling in the wind with my answers? Its like I never said anything......
June 26, 2008 6:46 PM | Reply | Permalink
Post a Comment