USC experts on the art and science of polling

Polling data saturates news coverage and political social media, and each source claims to represent the pulse of the nation, so how do you tell a good poll from a bad one? To answer this, USC experts break down the art and science of public opinion polling and the factors that determine a poll’s credibility.

Contact: Nina Raffio, raffio@usc.edu or (213) 442-8464; USC Media Relations, uscnews@usc.edu or (213) 740-2215

What are signs of a strong poll?

Experts say it depends on when the polling is done. During non-election periods, or when the race is many months away, strong polls require a few fundamentals: selecting unbiased samples, asking neutral questions, and disclosing methodologies, said Jane Junn, professor of political science and gender and sexuality studies at the USC Dornsife College of Letters, Arts and Sciences.

“When you’re thinking about calling races in a ‘horserace,’ one must consider not just how people describe their vote choice in the moment, but whether or not they are likely to turn out. In particular, you should look for how the pollster/outlet has defined the ‘turnout model,” she said.

“Constructing a turnout model is as much an art as it is science, with among the most influential elements in the latter being turnout among specific groups (i.e., young people, seniors, or Latinx people) in prior elections, and in this case, primaries in particular. But it is important that analysts model turnout with the proper comparisons, and in this case, primaries in a presidential election year.”

Contact: junn@usc.edu

What are some red flags to look out for?

“There are fake ‘polls’ that are not representative. These are any kind of ‘poll’ where you can participate and cast multiple votes or where the sample is not at all reflective of the overall electorate. These are Twitter polls or other surveys where the people taking it do not reflect the overall populace,” said Christian Grose, professor of political science and public policy at USC Dornsife and academic director of the USC Schwarzenegger Institute for State and Global Policy at the USC Price School of Public Policy.

“A second type of poll is one produced internally by campaigns. These polls are usually pretty good, but for the public we don’t often see those polls unless they are good for the candidate sponsoring the polls. In this instance, there might be several private public opinion polls conducted over the course of a candidate’s political campaign—but only one poll that happens to have the candidate leading is released to the media or to the public.”

Contact: cgrose@dornsife.usc.edu

Transparency matters

“Transparency is important for any type of research, not just polling. By sharing methods with others, they can verify the procedures we followed and perhaps detect errors,” said Arie Kapteyn, professor of economics and executive director of the Center for Economic and Social Research (CESR) at USC Dornsife.

Kapteyn oversees the Understanding America Study, a panel of households of approximately 14,000 representing the entire United States. Kapteyn and his team have made the majority of its data publicly available.

“The so-called ‘secret sauce’ that some organizations invoke as a description of their proprietary method is a recipe for abuse,” he said.

Contact: kapteyn@usc.edu

Additional Experts

Marc Baldassare is senior fellow, Bedrosian Center on Governance at USC Price. He is a leading expert on public opinion and survey methodology, and has directed the PPIC Statewide Survey since its founding in 1998.

Contact: markgbal@usc.edu

###

Wändi Bruine de Bruin is provost professor of public policy, psychology, and behavioral science at USC Price. During the 2020 presidential election, Bruine de Bruin co-developed an innovative polling method that asked people who their friends and family plan to vote for—a methodology that proved to be a much more accurate predictor of the election outcome than the traditional question of “who do you plan to vote for?”

Contact: wandibdb@usc.edu

###

Jill Darling is an expert in survey research, polling, public opinion and tracking polls. Darling is survey director at the USC Dornsife Center for Economic and Social Research (CESR).

Contact: jilldarl@usc.edu

###

Bob Shrum is a renowned political strategic and director of the USC Dornsife Center for the Political Future. Shrum is an expert in political campaigns, presidential elections and applied politics.

Contact: uscnews@usc.edu or cpf@usc.edu

###

(Photo/iStock)