Presidential Polls and the Quality of Marketing Research

At the time I am writing this post, it is October 18, 2012 – less than three weeks before the upcoming election. More than any other election I can remember, the 2012 presidential election has been characterized by a growing overabundance of polls, poll results, and survey-based predictions about which candidate will prevail on November 6.

Whether this is good or bad depends on your perspective. For me, someone who loves research, enjoys surveys, and is entertained by quantitative measures, the data is wonderful and fascinating. I find reading polls a bit like reading football or baseball statistics, except there is a lot more at stake than a single sports game or even a World Series title.

Some though, find the abundance of polling data overwhelming, and worry that it may distract their candidate, or even an opposing candidate, from following their true positions on key issues. Worries about polls are compounded by the lack of agreement from one poll to another, and even within some polls from week to week.

This post provides a primer on presidential polls to explain what is happening, addressing questions such as “Are there really a lot of polls or is it my imagination?” Or, “Why are there so many polls?” And, “Why don’t they all agree?”

The same trends that have made survey data increasingly available in presidential elections have also made survey data ubiquitous in many companies, through reports such as market research studies, customer satisfaction surveys, tracking studies, and market segmentation studies. Later in this post I’ll explain why a similar set of trends have made more data available in both contexts, and why, whether dealing with a presidential poll or a research study, it is more important than ever to understand how the data was gathered, and how to separate the trends from the background noise.

So many polls, so little time…

A complete list of polls that track the 2012 presidential election would be very long. For example, Reuters/Ipsos surveys 11,000 people per month, asking questions that cover the presidential race and a host of other issues.

The Pew Research Center for the People & the Press, which describes itself as both independent and non-partisan, regularly publishes poll results. Rasmussen Reports publishes a daily presidential tracking poll, which relies, for better or worse, on a survey of likely voters. CNN, not to be outdone, has its own Polling Center, with sections covering a wide variety of issues for the 2012 election in the many-windowed formats we have seen in some CNN broadcasts.

These days, the rising star of the polling world may be the New York Times’ FiveThirtyEightBlog. Named after the number of delegates needed to win the presidency, this blog provides data culled from a variety of sources, as well as some of the best writing I’ve seen anywhere on the subjects of polls and data analysis. Every day Nate Silver updates each candidate’s chance of winning, along with other predictions at the national and state level. (Well, almost every day. Mr. Silver did not provide an update yesterday because he was the featured guest on “The Daily Show With Jon Stewart.”) In the 2008 presidential election, Mr. Silver correctly predicted the winner of the Presidential contest in 49 of 50 states (missing only Indiana), and correctly predicted the winner of all 35 Senate races that year.

I could go on. This list is not exhaustive – there is also the Gallup poll, Washington Post/ABC News poll, the NBC News/Wall Street Journal/Marist poll, and many others.

Why are there so many presidential polls and why don’t they all agree?

There are, I believe a few reasons behind the increasing availability of survey data and polls in elections:

1. There are many more news outlets. The 2012 election is covered by broadcast networks, cable networks, online news organizations, and a host of other reporting entities. They all have space to fill, and they’ve found polling a useful and concise way to explain a presidential election, which involves very complex issues. Polling helps explain who won a presidential debate, or whose message is getting through.
2. People like it. Like it or not, I believe polling is here to stay because people enjoy the ease and simplicity of reading polls. I’m not the only one who enjoys watching polls the same way I enjoy watching sports scores. After a debate, I’m interested in finding out how the interaction I just watched affects likely voting patterns.
3. The internet has made it possible. The internet is a radical innovation that is still evolving and has changed polling in a number of ways. The internet makes polling much more feasible, because many polls are conducted online or augmented with online data. Also, the internet has shortened the news cycle – many of us now carry the internet in our phones or on our tablets, so we can check polling data in many more places, creating greater demand for polls.
4. The election is close. The election in 2012 has been relatively close, particularly since Mitt Romney’s campaign was re-energized with the first debate. Obama and Romney have generally been within a few percentage points of each other in the popular vote, and neither candidate has displayed an insurmountable lead at any point. Any close match is always more interesting to watch.
5. The campaigns have led the way. The campaigns were early to understand polls, and they know that, like it or not, polls matter. Campaign staff and spokespeople have led the way in using public opinion surveys to develop campaign and communications strategies, and communicating the results of polls to support their positions. Pollsters such as Frank Luntz have moved across assignments on behalf of campaigns, the media, and the private sector.

The presidential polls don’t all agree because they are not all conducted in the same manner. Some polls rely on likely voters, others rely on all respondents. Some polls are conducted online, others by phone. Different polls weight their data differently across demographic groups or across political parties. Timing also varies across polls, both in terms of when they are conducted and whether they rely on data collected in a short period of time (such as a single day or week) or rolling in over a longer period of time.

On top of this, each poll has its own margin of error based on factors such as sample size.
Given all these differences, it’s a wonder we don’t see more discrepancies across the various polls. You can read more about the differences across polls in a recent article from CBS News.

Shared lessons for presidential polls and commercial marketing research

My firm is in the business of providing organizations with data about their customers. Like presidential polls, in my business I’ve watched data become increasingly available in organizations through vehicles such as marketing research reports or customer satisfaction surveys. In fact, it is not just presidential elections and marketing research where data has become increasingly available – according to IBM, 90% of the world’s data was created in the past two years.

Data about markets and about customers is now available from a wide variety of sources, including marketing research reports, customer satisfaction surveys, social media, and all the customer data that companies now collect in the course of doing business. However, the core lessons about using this plethora of data apply equally well to presidential polls and marketing research.

As the availability of data has grown, it is not just the numbers that matter, but where they came from. Yes, it is increasingly easy to gather data, but are the numbers reliable? For example, pollsters and survey takers are finding today that consumers and customers are busier than ever, making it more important to have respondents who are engaged in the data collection process so that they provide meaningful results.

The wise consumer of information for presidential polls or corporate initiatives will pick and choose their sources carefully. Those who support one candidate can always find a poll that says their candidate has the momentum, their party is winning, or their initiative will pass. That doesn’t make it true. Similarly, managers can always find survey data that suggests customers are satisfied, product launches are succeeding, or brands are growing.

Occam’s razor suggests that in a sea of data, the best approach may be the simplest, finding a simple set of measures that are easy to understand, and can track or explain the election, market, or customers in question. In presidential elections, that has often been the President’s approval rating or the jobs reports. For some customer satisfaction studies, that measure has been Net Promoter. For some concept tests, it has been purchase intent.

Whether it’s a presidential election or corporate initiative, the core principles of marketing research are similar. Make sure you know the source of your numbers, and choose carefully among the many measures that may be available to you.