November 18, 2012 | 8:00 a.m.
Last week's presidential election has widely been seen as a victory for pollsters who, on balance, saw President Obama as the favorite before Election Day. But that wasn't the case for the esteemed Gallup Organization. Its polling showed Republican Mitt Romney with a significant lead among likely voters 10 days before Nov. 6 and marginally ahead of Obama on the eve of an election that Obama won by about 3 percentage points.
At an event on Thursday at Gallup's downtown Washington offices, Gallup Editor in Chief Frank Newport told a gathering of fellow pollsters that the organization was reviewing its methodology in light of these inaccuracies. But its fairly consistent Republican bias in 2012 and its overestimation of the white portion of the electorate raise important questions about sampling and the way Gallup determines which respondents are registered and likely to vote.
"We don't have a definitive answer," Newport said.
The day before Election Day, Gallup released data culled from the four previous days, showing Romney with a 1-point lead among likely voters, 49 percent to 48 percent. Before that final survey, Gallup had suspended polling for three days in the aftermath of Hurricane Sandy, when nearly 10 million Americans were without electricity.
Immediately before the storm hit, Gallup showed Romney ahead by 5 points, 51 percent to 46 percent, and Romney led by as many as 7 points in mid-October. All the while, most other national polls showed a neck-and-neck race.
The reasons for Gallup's inaccurate results remain unclear, and they are particularly baffling because Gallup, from a simple methodological standpoint, does things the right way. The company calls both landline and cellular telephones, and in October, it increased the proportion of cell-phone interviews to half. Moreover, Gallup remains among the world's most prominent and respected public-opinion organizations, and its more than 75 years of polling data comprise a large portion of the information we have about Americans' attitudes about their government and society over that time.
So were Gallup's struggles this year the result of sampling bias — through its random-digit-dialing interviews, did Gallup simply talk to too many Romney supporters? Was its likely-voter screen filtering out Obama supporters who would go on to cast ballots for the president? Or is it some combination of the two?
Gallup's likely-voter model is a battery of seven questions it uses to determine which respondents are most likely to cast ballots. These questions include how likely they say they are to vote, their self-reported vote history, whether they know where to vote, and how much thought they have given to the election.
Respondents are awarded points for their answers to these questions, and only those who accrue a significant number of points pass through the likely-voter screen. But these measures may have led Gallup to understate the participation of the critical demographic groups that comprised Obama's winning coalition: younger voters and minorities.
On Oct. 26, Gallup released a demographic analysis of those respondents classified as likely voters in its daily tracking poll between Oct. 1 and Oct. 24. Of those voters, 78 percent were classified as non-Hispanic white, significantly more than the percentage of white voters measured by exit pollsters, 72 percent. Four years ago, Gallup also found an electorate that was 78 percent white, an overestimation from the 74 or 75 percent recorded by exit polls. But this year's disparity is of a greater magnitude.
Gallup has also underrepresented younger respondents in its measures of likely voters. Over the first 24 days of October, 13 percent of Gallup's likely-voter sample was younger than age 30. Exit polls show these younger voters made up 19 percent of the national electorate.