Reader Posts

« previous | TPM CAFÉ READER POSTS HOME | next »

Polling.. are there any poll buffs out there who can illuminate?

avatar

How are to we treat/explain the extraordinary differences in different groups of polls?

Times/Bloomberg yesterday had Obama leading 49-37

Newsweek on the 20th  had him leading 51-37.

Gallup today has them tied.  Rasmussen has them 49-45 if you include leaners.  And at no stage over the last week have either of them come up with anything approaching the Bloomberg/Newsweek polls.

Why the amazing discrepancies?  What are we to deduce from them?   Is there any analytical/statistical reason that makes sense of this?


Comments (12)

For a good online place to read/discuss/understand polls, try Pollster.com a place for serious statistically-minded "geeks" (and I say that with respect and affection, being "poll-challenged" myself.)

Arrrrgh! damn links...

Pollster.com

and in the HIGHLY LIKELY EVENT the link fails:

http://www.pollster.com/

reezenfrazzithotdoggoneitdickdastardly!

UNCLE!...

Pollster dot com

I am going to go slit my wrists now with a very dull razor.

Goodbye cruel world....

avatar

LOL! Thanks, Jade. I did go there (and fivethirtyeight) before posting this question. Neither of them illuminated.
sigh

You might also try http://www.fivethiryeight.com

Excellent polling site

First, it helps to get a better picture of the polls by seeing more of them...

http://www.pollster.com/08-US-Pres-GE-MvO.php

Second, sample size and population are relevant. Pollster lists three types of population, RV, LV and A.

RV is registered voters, presumably who said yes to a question "Are you registered to vote?"

LV are "likely voters" most likely, people who answered yes to a question such as, "Did you vote in the last election?"

A is (my guess, I didn't read the whole site) all respondents.

Large sample size LV is the likely best survey. There are other characteristics that may not be reported such as whether the samples are clustered, the number of call backs (likely not many), the non-response rate, and for some, the use of internet and/or volunteer sampling (volunteer sampling is essentially invalid).

You will notice that there has not been that much change in the spread between the two candidates in the large sample size LV surveys over the past 20+ days (it ranges form 4 to 7 points).

The whole point of sampling is to REDUCE risk of error, not eliminate it. When you hear that the margin of error is +/- 4 points, the missing words are "with a x% confidence level." I don't know the industry standard for this purpose, but the universal standard is 5% unless the issue is "life or death." Five percent means one will be wrong, SUBSTANTIALLY WRONG, once every 20 times.

With an average of more than one poll a day, that would mean a substantial error (even without considering the RV, A issue) more often than once every 20 days. Over a longer time period, one would expect those errors to sometimes spread out and other times cluster.

avatar

um...

(pollstertwit here)

I did follow the link and am none the wiser.

Do you have a sense from all this which ones are the more credible? ie the dailies or those others which give him a big lead?

Yes, pollstertwit... READ WHAT I SAID ABOVE http://tpmcafe.talkingpointsmemo.com/talk/2008/06/polling-are-there-any-poll-buf.php#comment-2929433 . It lays it out for you.

I'm not a polling buff, but I read an explanation today at The Carpetbagger Report: http://www.thecarpetbaggerreport.com/archives/15988.html#more-15988

I didn't really understand it, but maybe it will make sense to you.

Ah, I see what is wrong with my answer. I didn't say the big spread is likely right. Well, my gut says it is, the math says it isn't. The one thing that is brought up at The Carpetbagger Report that I missed is the reweighting issue. If we think the shares of Dems and Reps are pretty much fixed, then reweighting is important, especially when polls don't do callbacks (I think). If we think that those shares are in flux, then reweighting can make the results unreliable. Too bad we don't know the answer.

Fran, my guess is that there are two explanations. The first is statistical and the other is methodological.

If five firms do a poll using the same methodology but asking different sets of randomly chosen respondents, they will come up with different numbers. Although the differences should be relatively small, within five percentage points or so.

The larger differences will be methodological. First, how the samples were selected and more importantly, how they were interpreted.

The way undecided voters are handled is clearly going to affect the final results.

In an "odd" year like 2008, things get complicated as it is harder for the pollsters to predict voting patterns. They do try to predict them, but different firms will have different guesses. Even with the same set of raw data, the interpretation may well be different.

In the US electoral system, it's all of course doubly complicated because of the Electoral College, where the number of votes is not as important as how they are distributed. National polls might even be misleading.

The bottom line is that we won't find until November who gets to be the next president :)

Was I whistling in the wind with my answers? Its like I never said anything......

Post a Comment

Inside Cafe



Cafe Features


October 20-24

Book Cover

October 27-30

Book Cover

November 17-21

Book Cover

December 1-5

Book Cover

Book Club Archive



Masthead

Editor-in-Chief
Josh Marshall

Site Editor
Lila Shapiro

Intern
Claire Wilcox



Subscribe to TPMCafe's feed.
Subscribe to TPMCafe's reader blog feed.

Advertise Liberally
Share
Close Social Web Email

"To" Email Address

Your Name

Your Email Address