The future of polling: Prediction or Observation?

Yeah, the "unskewed poll" predictions.

Yeah, the “unskewed poll” predictions.

For years, public opinion polls were, essentially, conducted in the same way. When a polling firm conducted a survey, it was all within the time frame of that poll. This was always qualified with the statement “if the election for President (or whatever) was held today…”. It was known that a poll only provided a snapshot of the electorate at that specific period of time, and did not predict future trends or eventual election results.

However, in the last few election cycles, we see a change in the way public opinion polling is conducted. Whenever a poll from a legitimate polling firm is released, the political pundits on the losing side of that poll always make the same argument, which is “(insert party here) voters were over-sampled”. It is this “over-sampling” which leads to pundits saying that the poll is absolutely baloney. However, these polls weren’t taken to make predictions, but to take that snapshot of the electorate.

However, it seems as if the pundits are looking for predictions, not random sampling. This has resulted in a new method where a polling firm “predicts” turnout for an upcoming election, and uses the polling result as an answer to a future question, such as “who will win (insert race here)”. While tradition opinion polls say “if the election were held today”, this new way of polling says “using samples from today, we predict…”.

So, what is the problem with this method? The main problem from the traditional survey researchers is that the results of the poll are no longer a random sample, when the party parameters are being set by humans. If John Doe predicts that 45% of the voters will be Democratic, but 56% of the sample is Democratic, John will weigh it so that Democrats only represent 45% of the sample. This can be done one of two ways. First, they can weigh the totals. The second way is that they can stop sampling after the 45% is obtained. Therefore, 11% of the Democratic sample in the first example would be rejected. Basically, it is adding human opinion into random sampling, which is usually frowned upon.

But should it be?

It all depends on what the poll is trying to accomplish. If a campaign is looking to do a benchmark poll, then going the traditional route would make the most sense. However, if a news media outlet is trying to “predict” and election, doing the predictive or “unskewed” method (which we will get to later) is the answer.

Another problem arises when we look at polls at the end of the election. We say that “polling firm ‘A’ was the most accurate this cycle.” But what does that mean exactly? How do we not know that a poll conducted in April was an accurate representation of the way things would be played out in April? The problem arises when we use polls that happened before the election, even months before the election, and compare them to the ballot results. That is not the goal of random sampling.

But if we were running a campaign, wouldn’t we want a poll that would best represent the climate on election day, which would weigh certain demographics? While it might not be entirely random, it could be a good method to make predictions.

However, if we are going to look at doing polling this way, the problem isn’t the lack of a random sample, but who is making the predictions as to eventual voter turnout. In 2012, the “Unskewed Polls” guy basically said that Democrats were over-sampled in every state, and that Romney would sweep the nation. This Republican bias enter into the equation and this methodology was extremely flawed.

But if a system was put in place that could provide sound methodology behind making predictions, would it help? I really see no reason why this would be the problem. However, in order to make this work, bias must be removed. In a pure random sample, you somewhat remove the bias, unless you are conducting a push poll.

This new polling methodology has been highly criticized, but maybe we need to give it a chance. But since the introduction of this new methodology, polling firms should now clarify whether the poll they are conduction is a “random sample poll” or a “prediction poll”. Both would provide vastly different results, but both could be quite beneficial.

With that being said, let’s not throw out the baby with the bath water quite yet. Let’s see how this works out. If bias is removed, it could be an accurate predictor of elections. However, pollsters will tell you that they aren’t in the prediction business…though many people think that they are. These new polling firms are trying to make polling a predictive practice.

Advertisements