The Crisis of Polling

By George Friedman

Last week, in the run-up to the first vote that would help determine the Democratic and Republican nominees for U.S. president, polls said that Donald Trump would win the Iowa Caucuses. He didn’t. There are many explanations for this, the leading ones being that his supporters were heavily tilted toward those who had never shown up at a caucus before, that the campaign’s “ground game” was off and that his supporters didn’t turn out to caucus. These are reasonable explanations and certainly part of the answer, but the issue of polling accuracy goes beyond Trump. British Prime Minister David Cameron was trailing, according to polls, prior to the election in 2015. He won a solid victory. Turkish President Recep Tayyip Erdogan was supposed to be in the fight of his life before the 2015 parliamentary election according to Turkish polls. He won a substantial victory. It also seemed that Israeli Prime Minister Benjamin Netanyahu would lose in 2015, and then he won. And in 2012, Republican presidential nominee Mitt Romney’s pollsters genuinely believed he would beat Barack Obama. He didn’t.

For many years, election polls were extraordinarily accurate. They may still have value, but they can no longer reliably predict which candidate will get the most votes on election day. It is important to understand why this has changed and how it impacts politics, bearing in mind that this is a global, not solely American, phenomenon.

Polling depends on three things. First, a detailed model of how people voted in previous elections in discrete geographical areas. Second, there must be a mathematical model for comparing current polling results with historical patterns. Finally, there must be a method for collecting data based on where the voters will vote. For a long time, the focus has been on the mathematical model. The basic data set had sufficient details and was the template against which to measure current data, and the mathematical model did the measuring and compensated for shifts. Many pollsters would say that this is too simplistic and I have missed many details. They’d be right, but what I’m trying to do is get a sense of why polling results have become so unreliable.

For years, the baseline used for U.S. polls was election results on a precinct level plus social and economic data drawn from the U.S. census or equivalent sources around the country. Using this data, pollsters could determine key social traits of the area, from the dominant religion to the average age of residents. Then, the polling organizations would call people in the area and ask them questions. This was relatively simple. Landline phones allowed pollsters to determine the location of the polling respondents.

Over the past decade, the number of landlines decreased dramatically as they were replaced by cell phones. The problem is not that cell phone numbers are hard to find, but that they don’t tell you where the person who owns the number is located. I have a son who went to school in St. Louis and graduated some 15 years ago. He lives in California now. But he still has a 314 St. Louis area code. If pollsters call him, they will have to rely on his word to determine his real place of residence, which does not match the area code of his phone number. With landlines, pollsters were able to tell where the phone and its owner were located. And therefore they knew what geographical area they were polling.

There are certainly people who still have landlines, but they are socially skewed. The older you are, the more likely you are to have a landline. The same is true to a lesser extent for the less affluent. If you sampled a population in a given region using landlines, your results would be skewed massively. I wonder whether the overestimation of Trump’s strength in Iowa had something to do with pollsters using landlines and sampling older and less affluent voters, who are exactly the group of people supporting Trump. In any event, cell phones have transformed how polling agencies can reach people dramatically.

The second problem is the data set. Prior to the past few decades, there had been continuity in society for the most part. Since the 1970s, after the upheavals of the 1960s, society has shifted moderately from election to election. The upheavals following the 2008 crisis may have brought about another fundamental shift globally. This means that the voting patterns of a 45-year-old professional or a 60-year-old blue-collar worker have shifted and the prior model no longer applies. Therefore, it is unclear whether prior data sets are predictive for contemporary elections. Before this social shift, a 45-year-old professional might have tended to vote a certain way. It isn’t clear whether this is still the case. And this is the core problem with all sampling. Pollsters find out that their model doesn’t work only after they get wrong results.

Finally, there is a global distrust of institutions and how they will use data. Many who don’t want their voting plans revealed, and do not trust the promise of anonymity from the pollster, may give answers that are false when asked who they will support on election day. Notice that Ted Cruz, Netanyahu, Cameron and Erdogan can all be called conservative. All were expected to lose. None did. There is a phenomenon in which people say what they think pollsters want to hear or refuse to reveal beliefs they think are unpopular or looked down on. In many societies, conservatives are regarded as reactionary and socially obnoxious, particularly by the educated. Voters will perceive pollsters as being part of this educated class, whether they are or not. Therefore, they won’t say things they think will be held in contempt. But they will vote that way. This may have nothing to do with why some polls have been so inaccurate in recent years, but I do see a pattern of conservatives doing better in elections than polls have predicted.

There are multiple things that are going wrong, and in my view it is the switch to the use of cell phones on a massive scale and social turmoil that limit the value of the templates against which current polls are measured. The polls no longer accurately predict election outcomes. And that will have a profound influence on politics.

Modern politics is the politics of marketing. The marketing strategy is to tailor the message to the buyer’s views. The power of marketing was knowing what sells. If the polls are all over the place, the politician has no reason to listen to the marketer. Consequently, two things will happen. First, politicians may actually say what they think, since they have nothing to lose. Even more, intuitive politicians, like Franklin D. Roosevelt, Harry Truman and Dwight Eisenhower, might have a huge edge over packaged politicians. Even if they followed public opinion, an intuitive understanding of public opinion, rather than one filtered through the screen of polls and marketing experts, might be beneficial.

At the very least, we seem to be entering a period where polls give us no more than a broad sense of what is happening, and not enough to guide candidates’ positions, which has been increasingly apparent in the past half century. In every case I cited, the polls did not tell us what would happen in the election and therefore could not tell us what mattered to voters. If poll results are not reliable, then the candidates are left to speak their own minds, or more likely, rely on the instincts of old-style bosses, who guided them. In the end, we will not know the outcome of the election until the votes are actually counted.

But undoubtedly a new methodology for polling will emerge, either going door-to-door or using drones to listen in on people’s conversations. In any case, for now, the polling system is clearly weak. This uncertainty means that election night actually is more important than the latest poll from a newspaper or TV network. Although I have been referring mostly to the U.S. polling system, the same applies for Britain, Israel and Turkey. Clearly, this is not an American problem alone. It applies also to democratic countries that are more volatile. An incorrect poll can raise questions about the legitimacy of the counting of the vote, which can turn this problem into an explosive issue.