The outcome of the 2016 presidential election shocked many people – and they pointed their fingers at misleading polls that didn’t do a great job predicting what actually happened.

On Election Day, analyst Nate Silver’s FiveThirtyEight predicted that Clinton had a 71.4 percent chance of winning and ran the headline “Final Election Update: There’s a Wide Range of Outcomes, and Most of Them Come Up Clinton.” Economist David Rothschild’s PredictWise site gave Hillary Clinton an 89 percent chance that morning. And The New York Times’ The Upshot set her chances at 85 percent, suggesting that “Mrs. Clinton’s chance of losing is about the same as the probability that an N.F.L. kicker misses a 37-yard field goal” – a low probability event that happens only one or two times out of 10.

But when the votes were counted, expectations were upended and Donald Trump won. How did so many polls seemingly miss the target? As a behavioral scientist, I study people’s emotions and how they affect behavior – and I suspect emotions might have something to do with why the polls were so mistaken.

What’s behind bad predictions

There are several possible reasons for the misleading poll predictions, and of course a mix of all these issues might have occurred. Maybe the polls underestimated what turnout would be among demographic groups (such as white voters) who disproportionately supported Trump. Perhaps Trump supporters were less likely to respond at all to pollsters given how much their candidate attacked polling, calling it rigged. That would leave them underrepresented in the poll data.

It’s also possible polling didn’t account well for what “undecided” respondents would do once in the voting booth – one conjecture is that Trump wound up with more votes from this group than expected. I think this last option raises an important possibility for what was going on: false data in the polls.

Predictions can be only as good as the data that support them; normally, data have no reason to lie. Polling data are based on voters answering strangers’ questions about who they will vote for, and generally, at least in the U.S., there is no reason to hide the truth. One is typically proud to vote for the candidate of one’s choice. Under normal conditions, when citizens are approached by pollsters about their voting intentions, the assumption is they’ll truthfully respond.

In this election, however, I suspect many people who said they were undecided were actually already committed to Trump. Societal pressures might have kept them from declaring their true intention. When surrounded by a perceived majority that favored Clinton, declaring that they themselves would vote for Trump might seem disgraceful and cause emotional discomfort. Swimming against the stream can cause distress.

Social norms and private embarrassment

Everyone has felt embarrassed at one time or another. It’s that unpleasant sensation we feel when we violate a social norm – for example, if your zipper is down in public, or you stumble down the stairs while people are watching. Traditionally, psychologists have thought of embarrassment as something that happens in public, in front of others who are taking note of our transgression or blunder.

But in my research, I’ve found that even actions made in private can lead to embarrassment, if one imagines someone watching. For instance, buying Viagra, even from the privacy of one’s home, can lead some people to feel embarrassed. They believe unenhanced sexual performance is the social norm. They imagine other people observing them violate this expectation and judging them, leading to embarrassment for something they’re doing in private.

Back to the polling situation. The norm in one voter’s social milieu may be voting for Clinton. A vote for Trump would defy that social norm. As such, even if the polling is confidential and private, a citizen may not want to disclose to a pollster his intention to vote for Trump. He’d rather say he’s “undecided” and spare himself the discomfort of violating a social norm. If the poll is done over the phone, a less anonymous venue than an online survey, for instance, then this embarrassment is more acute and leads to a greater likelihood of a Trump-leaning respondent claiming she’s undecided or even that she’ll vote for Clinton. Intending to vote for Trump becomes literally unspeakable.

Trump appealed to different people for a variety of reasons, which would influence how secretive they might be about their candidate preference. In rural America, people may have voted for him because of bad economic circumstances or a lack of confidence in the way different institutions currently function. Nothing too embarrassing behind that. Middle America proudly supported Trump.

But others – namely college students – may have been attracted to Trump because of their own fear of people of different races or from other countries. Racism and xenophobia are clearly against campus social norms. American universities reflect liberal values and expect students to embrace people from all countries, of all races and of all sexual identities. As such, students who reject this norm may hide that fact, and may also be embarrassed, ashamed or just plain scared to display their beliefs by publicly acknowledging support for Trump. If they are polled, especially in a non-anonymous phone interview, they may not declare their intention to vote for him.

Anonymity allows for true intentions

This hidden prejudice, and the social stigmas attached to it, may have been one leading cause of the bad predictions for white college graduates, and for those who were undecided. Support for Clinton may have been exaggerated among both the undecided and even those who declared they’d vote for her.

This phenomenon is similar to what pollsters call the Bradley effect. Los Angeles Mayor Tom Bradley, an African-American, lost the 1982 California governor’s race, despite leading in pre-election polls. Pollsters suspect that in races between a white and a nonwhite candidate, some voters who intend to vote for the white candidate tell pollsters they’re undecided or that they intend to vote for the nonwhite candidate. This happens because of social desirability – one wants to appear to do what is the socially correct thing to do.

This logic is corroborated in different ways. At the end of the summer, FiveThirtyEight identified that Clinton’s lead over Trump was wider in telephone surveys but couldn’t be sure why.

The USC Dornsife/Los Angeles Times poll was an outlier among other polls, consistently placing Trump in the lead. It relied on an ongoing internet survey. As an LA Times reporter put it on election night:

Trump voters were notably less comfortable about telling a telephone pollster about their vote. Voters who backed a third-party candidate were even less comfortable responding to a poll. Women who said they backed Trump were particularly less likely to say they would be comfortable talking to a pollster about their vote.

The feeling of anonymity offered by online, as opposed to phone, interviews allows for more honest revelation of a Trump vote. One less easily imagines someone judging them for violating the social norm.

To further test this logic, I asked my undergraduate class the day after the election whether they knew people who had voted for Trump but pretended they had not. There was a large show of hands. Here at the University of Michigan, Ann Arbor, the undergraduates are mostly white, and of course, college-going (a group that was mispredicted). My students said they knew of classmates and housemates who had not only voted for Trump, but had even pretended to have voted for Clinton, the popular candidate in their social circle. Some Trump voters had watched the election returns with friends and cheered along with the others whenever Clinton appeared to be doing well. Somehow news had spread about individuals who’d voted for Trump (or students could guess by the Trump supporters’ somewhat awkward behavior).

Social emotions distort the data

Obviously untruthful responses in polls skew the data, leading to inaccurate predictions. The numbers the pollsters obtained for some undecided voters and some committed Clinton voters appears to have been false data – accepted as correct, but in actually not. People’s embarrassment, shame and fear about showing support for Trump, especially if driven by misogyny, racism or xenophobia, made them lie in the polls and distorted the data.

It is tough to check if this hypothesis is true – does one call the people who said they were undecided and now ask them if they were actually committed to Trump at that time? After Trump’s win, they may not face the same societal pressures – would they now answer truthfully?

The next time around, pollsters may need to try harder to account for human psychology. Embarrassment, social desirability and fear of social stigma all have their role in human behavior during elections and can distort results if not acknowledged.

Aradhna Krishna, Dwight F Benton Professor of Marketing, University of Michigan

This article is republished from The Conversation under a Creative Commons license. Read the original article.