Nate Silver: polling problems
Earlier this morning I wrote about the fact that even Nate Silver, pollster extraordinaire, got this one wrong, and by a significant amount. I was thinking as I wrote that post that pollsters had gotten the Israeli election wrong as well. Both were predicted to be very close; both were not, in particular the UK election.
What went wrong with the polls? Now Nate Silver himself asks that question. Ever since the election of 2012 in this country, I’ve come to respect Silver as a pollster, because despite all the criticism he got that year, he was spot on. Here’s what Silver says:
Perhaps it’s just been a run of bad luck. But there are lots of reasons to worry about the state of the polling industry. Voters are becoming harder to contact, especially on landline telephones. Online polls have become commonplace, but some eschew probability sampling, historically the bedrock of polling methodology. And in the U.S., some pollsters have been caught withholding results when they differ from other surveys, “herding” toward a false consensus about a race instead of behaving independently. There may be more difficult times ahead for the polling industry.
I remember that, in the leadup to the election of 2012, a lot of conservatives were mocking the skewed polls, pointing out the cell phone problem, and in general saying the polls were biased. After the election, when the polls turned out to have been pretty accurate (and Silver extremely accurate), that put some of those arguments to rest. Now the arguments have woken from their slumber, making us wonder again—and making Silver wonder, as well.
Interesting.
Minor correction: Silver is not a pollster, but a modeler, who uses polls, as many (most?) modelers do.
What he is saying is that the people who collect essential data for him (the pollsters) are failing him.
(In principle, he could find some way to correct for sometimes systematic errors in the polls, but that is not an easy thing to do.)
One of the articles I read discussed a lot of swing/independent voters making last minute decisions…I don’t know how you can account for that in a poll.
I wonder if some of the way-off polls are not about data collection and prediction, but instead are intended to influence the election results. “No sense bothering to vote. Your side is gonna lose big-time.”
Another slightly less sinister possibility is that conservative voters refuse to talk to pollsters, regarding them as working for the other side.
I shouldn’t wonder if SteveS is right on both points – that poll results are being used to “push” the election to one side or the other … and that conservative voters are opting out of answering pollsters’ questions, either as a case of “none of your &$^@ing business” or — what is slightly more sinister — because your political inclinations toward a conservative candidate or cause can be used against you.
I look up telephone numbers all the time, using various web directories. I can often tag an address, sometimes a name to a telephone number. If I can – and I am usually trying this for innocuous purposes – what would stop someone looking to intimidate the subject of a supposedly neutral poll from doing the same, if they gave the “wrong” answer.
Alas, the world we have come to live in. Thanks, Obama, you have fundamentally changed America. Not much for the better, it seems.
At Econlog (an economics blog that leans libertarian), Scott Sumner has written a piece titled “There’s no such thing as public opinion.” (http://tinyurl.com/l8f6mzk)
Sumner’s argument is that almost all opinion polls are distorted by the way the questions are posed — i.e. framing.
Sumner doesn’t take on the obvious follow-up questions: have polls become another left-wing tool for influencing public opinion? Are they, in that sense, analogous to the infamous fact-checking sites? Is there a way to measure this bias?
Bottom line, public opinion polling is so erratic nowadays because of the diminishing ability to get an adequate random sample.
All I know is that over the years my message machine has recorded a number of attempts on the part of pollsters or political operatives to get my opinion. I have never replied.
Nearly a decade ago I got a letter from the Democratic National Committee soliciting my opinion on a number of issues. The questions were slanted. For example, there was a question about leaving Iraq, with all the choices put into time frame- leave in 6 months, a year, etc. There was no choice about “we leave when we win.” The letter ended with a request from funds. A year later, I got the same letter. This time I replied- not with answers but with a statement that as a former Democrat, I would just as soon die as vote Democrat.
With the election of Obama to the Presidency, I became more cautious- perhaps from his “joke” about auditing his enemies. I got some letters soliciting funds for the Obama campaign- well after he was elected- and didn’t reply. In effect, I decided I didn’t want t get audited.
The 2012 Selection was the long con — and in keeping with both Google and ayatollah Soetoro’s track record.
The W I D E O P E N nature of the MSFT 2004 operating system used in certain critical voting locations is now out and on the record.
For those with long memories: Florida was considered so ‘out of play’ that both campaigns largely folded up shop ten days before the vote.
Soetoro, himself, gave a pained, and painful, last minute — ill attended — sweep through Florida in the lead up to the election. All during the prior week, Florida had been polling ever further away — towards Mitt Romney.
Then, Tuesday, Miami kept coming up with last minute votes — over and over and over and over every time the (Republican) panhandle (one-time zone later to close: 60 minutes) — within seconds of EVERY panhandle update.
The Florida vote was HACKED. Period. Stop.
The Ohio vote was HACKED.
The Pennsylvania vote was HACKED.
Even North Carolina was HACKED. It wasn’t even slightly ‘in play’ but by the time the Black Hats were done — it almost jumped the full 8% into the Democrat column!
Yes, yes, ALL of those North Carolina votes keep trickling in from the urban ghettos until every cadaver had voted. The eastern most part of the state — closest to voting central — were the VERY LAST to tabulate their vote.
And this was in a state that went for Romney… just barely.
North Carolina was being kept in reserve until it was obvious that other stolen states would throw the prince over the top.
&&&&
Did one really have to await Buchenwald to know that Nazi Germany stunk?
In corruption of such a kind, of course you’re not going to get prompt admission of the super-crime.
It may not be “Chinatown” — but it certainly is Chicago on the Potomac.
Don’t wait for J. J. Gittes (Jack Nicholson) to check the plats to get the lay of the land on this one.
Remember the iconic photo of Truman holding up a newspaper whose headline, in huge font, was
“DEWEY WINS”?
As late as 1948, there was a demographic skew in who had phones by which to be polled, or who answered them.
Could be called The Bradley Effect.
Some years ago, a writer at National Review called a Brit election for conservatives. He said that when the nice middle-class person with the nice middle-class accent calls you, you certainly don’t want to admit you’re one of those slavering subhumans always slagged by the media and the beeb and comedians, intending to vote conservative. So you lie.
I think the point I was trying to make is that in recent years, long past Truman and Dewey, pollsters had become a lot more sophisticated in correcting for those sorts of errors. I kept hearing people say that the cell phone thing skewed sampling, and that people were lying to pollsters. But then when elections came round the polls usually were spot on, so those factors didn’t seem as big as people had been saying. But lately the polls really are off.
Polls are propaganda cloaked in a sugary paste for the common masses to consume.
But then when elections came round the polls usually were spot on, so those factors didn’t seem as big as people had been saying. But lately the polls really are off.
That’s what the fake election results were designed to emulate.
Correlation is not causation, remember that? Just because the poll numbers line up, does not mean the actual numbers lined up to the models of the polls.
The Powers That Be, if they’re smart, have already moved beyond polling. The alliance between Big Government and Big Data was purposeful. I wouldn’t be surprised in the least if the NSA were feeding data to Google so they could number-crunch for the Democrats.
Before you think this is tinfoil-hat stuff, read this article. It will make your hair stand on end:
http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html?pagewanted=1&_r=2&hp
All of this is cutting edge behavioral psychology. With enough data, a program can predict what you’re going to do before you even know, with a high (90+%) degree of accuracy.
That solves the problem of inaccurate polling right there. Of course, “they” don’t want you to know this.
Seriously, read that NYT article.
Ymarsakar:
I knew someone would bring up the “fake votes” bit. But I’m not ignoring that (in spite of the fact that I don’t think faking is as widespread as many people do, although I think it happens at times, especially in certain areas in close races). My point is still relevant, though, because why aren’t they faking things as well as before, then? Something seems to have changed re polling and elections.
neo
Wrt Truman and Dewey. Point is there’s likely to be something out there, something you don’t know about or haven’t thought about which will skew the results.
The world is full of hipsters up to middle-aged folks tripping over cracks in the sidewalk while drowned in some electronic device. Nobody knows whether they are more or less likely to answer the poll–if all you have is a land line and you get one conversation a day, a poll might be a fun distraction–more or less likely to lie, more or less likely to vote, more or less likely to lie about voting, more or less likely to vote on the subjects polled versus other issues.
I know people who are forever on their devices ordering their lives; calendars, appointments, reminders, tasks. Others are forever talking about what, if what I overhear is representative, amounts to nothing. Some of the latter conversations sound as if there’s some atrophied cognitive ability. Vote?
And, as with the Bradley Effect, there may be some lying for purposes of self-protection.