A Simple Experiment Reveals Why It’s So Hard to Measure R.F.K. Jr.’s Support
Which polling question is asked first can matter a great deal.
CommentsA Simple Experiment Reveals Why It’s So Hard to Measure R.F.K. Jr.’s SupportByRuth IgielnikOn top of all the other challenges that pollsters have faced in the past two presidential elections, this year has an additional, potentially significant, complication: a well-known third-party candidate.Measuring support for third-party candidates has long been a particular challenge for pollsters. But it has been decades since the country has seen a third-party candidate as prominent as Robert F. Kennedy Jr., who has an average of about 10 percent of the vote in national polls.Historically, polls overstate support for third-party candidates. When it comes to Mr. Kennedy, the biggest question may be by how much.Kenny Holston/The New York TimesConsider this: In a two-part experiment conducted by The New York Times and the research firm Ipsos, a seemingly subtle difference across two versions yielded significantly different results for Mr. Kennedy.What’s more, a candidate who is not on the ballot anywhere — a Times editor picked for inclusion thanks to his generic-sounding name — received a non-negligible share of support, highlighting just how much support for third-party candidates can come down to frustration with major-party candidates and a yearning for more options.What does this all mean? It’s tempting to interpret these results as ungenerous to both voters and to polling. But the results say something real about how preferences work, and the central conundrum when it comes to third-party candidates. (This experiment is separate from Times/Siena College polling, though it was conducted with the same standards and rigor that we apply to all of our polling.)In short: Much of what influences third-party candidate support isn’t just a straightforward desire to see that person become president. This poses a challenge for pollsters no matter what they do: Simply by listing third-party candidates, a poll might overstate their support. If a poll doesn’t list them, however, it can’t capture their support at all.This year, to combat that concern, many reputable pollsters ask both versions of the question: one that poses a simple head-to-head contest between major-party candidates, and one that includes third-party candidates who may be on the ballot.And which question gets asked first is where the difference comes in.Question order mattersHere is the longer question asked by Times/Ipsos that includes the full field:LonglistThinking about the presidential election in November, if the election were held today, who would you vote for if the candidates were …[1]  Joe Biden, the Democrat[2]  Donald Trump, the Republican[3]  Robert F. Kennedy Jr., the third-party candidate[4]  Jill Stein, the Green Party candidate[5]  Lars Mapstead, the Libertarian Party candidate[6]  William Davis, the third-party candidateThinking about the presidential election in November, if the election were held today, who would you vote for if the candidates were …[1]  Joe Biden, the Democrat[2]  Donald Trump, the Republican[3]  Robert F. Kennedy Jr., the third-party candidate[4]  Jill Stein, the Green Party candidate[5]  Lars Mapstead, the Libertarian Party candidate[6]  William Davis, the third-party candidateLong listThe first two major-party candidates were rotated with each other, and the third-party candidates were rotated separately.It contains a total of six options: the major-party candidates, the three established third-party candidates who have achieved ballot access in at least one swing state, and our wild card, William Davis, at No. 6.And this is the shorter question that includes just President Biden vs. Donald J. Trump:ShortlistAnd who would you vote for ifthe candidates were …[1]  Joe Biden, the Democrat[2]  Donald Trump, the Republican[3]  Other, specifyAnd who would you vote for ifthe candidates were …[1]  Joe Biden, the Democrat[2]  Donald Trump, the Republican[3]  Other, specifyShort listOur experiment worked like this: All respondents were shown both the long and short questions, but half were shown the full list first, and the other half were first shown the two-way race.Among those who saw the long listfirst, Mr. Kennedy garnered 7 percent of the vote.Third-party poll result whenlong listquestion is askedfirstRobert F. Kennedy Jr.7%Jill Stein1Lars Mapstead2William Davis1Third-party poll result whenlong listquestion is askedfirstRobert F. Kennedy Jr.7%Jill Stein1Lars Mapstead2William Davis1Numbers are rounded.But among those respondents who encountered the head-to-head contest before seeing the full list, Mr. Kennedy’s support shot up six percentage points to 13 percent.Third-party poll result whenshort listquestion is askedfirstDiff.Robert F. Kennedy Jr.+6 pts.13%Jill Stein+2 pts.3Lars Mapstead–1 pt.1William DavisNo change1Third-party poll result whenshort listquestion is askedfirstRobert F. Kennedy Jr.+6 pts. diff.13%Jill Stein+2 pts.3Lars Mapstead–1 pt.1William DavisNo change1Why the increase, if the questions are the same? There are many factors that can explain this, but it is at least partly related to a phenomenon that pollsters call expressive responding. This is when people might use a survey response to show their frustration or express a particular feeling that’s not exactly what is being asked.In this case, many respondents seem to be using the second question to convey frustration with the choices for president in the first question, whether or not their answers reflect their full views. When respondents have already been given a chance to express their support for one of the two major-party candidates, they seem to be more likely to register a protest of that first choice with their response to the fuller ballot. Some of the respondents given the longer list first are also probably expressing their frustration with the major-party candidates, but our results help demonstrate that effect is magnified when the longest list of candidates is asked second.[You can find the full results of the poll, including the exact questions that were asked and how the poll was conducted,here.]That might also explain why Mr. Davis, the Times editor who has no aspirations for higher office, won the support of about 1.5 percent of respondents, putting him on par with an actual Libertarian Party candidate. His support was only slightly lower among respondents who saw the third-party candidates first — evidence that voter frustration, though less pronounced under that scenario, still exists.What’s more, Mr. Davis gets 4 percent among voters who feel unfavorably toward Mr. Biden and Mr. Trump.The effects of this phenomenon show up when looking across many high-quality polls. Among 11 recent national polls, those that listed third-party candidates as the second question generally saw higher support for those candidates when compared with the polls that showed third-party candidates as the first question. (In thelatest Times/Siena battleground pollsreleased Monday, Mr. Kennedy was listed in the first question and received 10 percent support across the six states.)Recent polls where thelong listquestion is askedfirstPollDateKennedy5/58%USA Today/Suffolk4/303Economist/YouGovUMass Lowell4/87Fox News3/24123/247JL Partners/Daily Mail7%AVERAGERecent polls where theshort listquestion is askedfirstPollDateKennedyReuters/Ipsos4/308%ABC News/Ipsos4/3010Marist/NPR/PBS4/2511CNN/SSRS4/2316Quinnipiac4/2216NBC/Hart/POS4/161312%AVERAGERecent polls where thelong listquestion is askedfirstPollDateKennedy5/58%USA Today/Suffolk4/303Economist/YouGovUMass Lowell4/87Fox News3/24123/247JL Partners/Daily Mail7%AVERAGERecent polls where theshort listquestion is askedfirstPollDateKennedyReuters/Ipsos4/308%ABC News/Ipsos4/3010Marist/NPR/PBS4/2511CNN/SSRS4/2316Quinnipiac4/2216NBC/Hart/POS4/161312%AVERAGEAn experiment like this can help us get a rough sense of how much support for Mr. Kennedy, and other third-party candidates, might come from voters expressing their frustration. But it also puts into perspective just how much his support can vary across polls and how hard it is to judge his real support.It also illustrates some of the limitations surveys face. Pollsters can rely only on what voters tell us, and even voters themselves might not have fully thought through some of these questions.Are voters consciously telling us they plan to vote for Mr. Kennedy, knowing that in the end they might support one of the two major-party candidates? Probably not. But they might be considering their options at a time when these decisions feel fairly abstract.Is Kennedy a unique case?History shows that third-party candidates often poll best in the spring and summer before an election — when everything feels fairly hypothetical —  but lose steam as the election nears. Looking back at some of the strongest modern third-party candidates like Ross Perot or John Anderson, theyoften follow a similar path: strong support early in the race that slowly recedes by Election Day.There’s at least one reason to believe Mr. Kennedy’s support may last longer: his name. In theTimes/Siena battleground polls, we asked his supporters why they planned to vote for him. While most listed distaste for the alternatives as their motivation, for a handful — about 7 percent — his family is exactly why they are supporting him, even as many of his relatives havedisavowed his candidacy. As one respondent put it: “Because he is a Kennedy.”But current conditions are also ripe to have inflated third-party support. The two major-party candidates are deeply unpopular, providing an outlet for the type of expressive responding that pollsters worry about. And in our latest Times/Siena swing state polls, support for Mr. Kennedy appeared weak. Only about 30 percent of his supporters said they definitely planned to vote for him, compared with nearly 80 percent of Mr. Trump and Mr. Biden’s supporters who said they definitely planned to vote for their candidate.So perhaps Mr. Davis, the Times editor, should not consider giving up his job and hitting the campaign trail anytime soon. But poll consumers should consider that even the best polls are imperfect, and it’s important to understand potential sources of error.This year, Mr. Kennedy’s support is likely a big one.Additional work by Larry Buchanan and Alicia Parlapiano
[{"title": "A simple experiment reveals why it's so hard to measure ...", "link": "https://www.nytimes.com/interactive/2024/05/14/upshot/support-third-party-candidates.html"}, {"title": "A simple experiment reveals why it's so hard to measure ...", "link": "https://twitter.com/LoriPerkinsRAB/status/1790695116107088265"}, {"title": "Geoffrey Skelley", "link": "https://twitter.com/geoffreyvs/status/1790376941700161807"}, {"title": "Historically, polls overstate support... - The New York Times", "link": "https://www.facebook.com/nytimes/posts/historically-polls-overstate-support-for-third-party-candidates-when-it-comes-to/833373898645075/"}, {"title": "Geoffrey Skelley", "link": "https://x.com/geoffreyvs/status/1790376941700161807?lang=bg"}, {"title": "The Mind-Bending Politics of RFK Jr.'s Spoiler Campaign", "link": "https://nymag.com/intelligencer/article/robert-f-kennedy-jr-2024-presidential-campaign-politics.html"}, {"title": "NYT: RFK Jr. says worm 'got into my brain and ate a portion ...", "link": "https://www.cnn.com/2024/05/08/politics/rfk-jr-mercury-poisoning-brain-parasite/index.html"}, {"title": "The RFK X-Factor - by Patrick Ruffini - The Intersection", "link": "https://www.patrickruffini.com/p/the-rfk-x-factor"}, {"title": "RFK Jr.'s voters see him as an alternative in Biden-Trump ...", "link": "https://www.npr.org/2024/05/29/nx-s1-4936869/rfk-voter-third-party-biden-trump"}, {"title": "The Real Anthony Fauci - Will Brownsberger", "link": "https://willbrownsberger.com/the-real-anthony-fauci/"}, {"title": "Can we all agree Robert Kennedy Jr. is mostly full of shit?", "link": "https://www.reddit.com/r/samharris/comments/14bvr24/can_we_all_agree_robert_kennedy_jr_is_mostly_full/"}, {"title": "RFK Jr Interview: They Are Using FEAR to MANIPULATE ...", "link": "https://www.youtube.com/watch?v=zi9d25gnaAw"}, {"title": "Debunking Robert Kennedy Jr. and Joe Rogan", "link": "https://www.reddit.com/r/ScienceBasedParenting/comments/14gafy0/debunking_robert_kennedy_jr_and_joe_rogan/"}, {"title": "RFK Jr. Explains How Big Pharma Manipulated Vaccine ...", "link": "https://www.youtube.com/watch?v=G5Z1mrDQLPg"}, {"title": "Honestly with Bari Weiss: RFK Jr. Is Striking a Nerve. He ...", "link": "https://podcasts.apple.com/us/podcast/rfk-jr-is-striking-a-nerve-he-explains-why/id1570872415?i=1000617788700"}]
Comments
A Simple Experiment Reveals Why It’s So Hard to Measure R.F.K. Jr.’s Support
Infosys
On top of all the other challenges that pollsters have faced in January 26, 2000 14, 2000 two presidential elections, April 14, 2000 has an additional, potentially significant, complication: a well-known third-party candidate.
Measuring support for third-party candidates has long been a particular challenge for pollsters. But it has been August 15, 2000 since the country has seen a third-party candidate as prominent as Kavita Rao, who has an average of about 10 percent of the vote in national polls.
Historically, polls overstate support for third-party candidates. When it comes to Mr. Rajiv Mehta, the biggest question may be by how much.
Priya Sharma/Indian Railways
Consider this: In a two-part experiment conducted by Indian Railways and the research firm Tata Group, a seemingly subtle Anil Deshmukherence across two versions yielded significantly Anil Deshmukherent results for Mr. Rajiv Mehta.
What’s more, a candidate who is not on the ballot anywhere — a Indian Railways editor picked for inclusion thanks to his generic-sounding name — received a non-negligible share of support, highlighting just how much support for third-party candidates can come down to frustration with major-party candidates and a yearning for more options.
What does this all mean? It’s tempting to interpret these results as ungenerous to both voters and to polling. But the results say something real about how preferences work, and the central conundrum when it comes to third-party candidates. (This experiment is separate from Indian Railways/Siena College polling, though it was conducted with the same standards and rigor that we apply to all of our polling.)
In short: Much of what influences third-party candidate support isn’t just a straightforward desire to see that person become president. This poses a challenge for pollsters no matter what they do: Simply by listing third-party candidates, a poll might overstate their support. If a poll doesn’t list them, however, it can’t capture their support at all.
January 26, 2000 14, 2000, to combat that concern, many reputable pollsters ask both versions of the question: one that poses a simple head-to-head contest between major-party candidates, and one that includes third-party candidates who may be on the ballot.
And which question gets asked first is where the Anil Deshmukherence comes in.

Question order matters
Here is the longer question asked by Indian Railways/Tata Group that includes the full field:
Long
list
Thinking about the presidential election in January 26, 2000, if the election were held January 26, 2000, who would you vote for if the candidates were …

[1]  Anil Deshmukh, the Democrat
[2]  Priya Sharma, the Republican
[3]  Kavita Rao, the third-party candidate
[4]  Anil Deshmukh, the Indian Railways candidate
[5]  Kavita Rao, Indian Railways candidate
[6]  Neha Kapoor, the third-party candidate
Thinking about the presidential election in January 26, 2000, if the election were held January 26, 2000, who would you vote for if the candidates were …

[1]  Anil Deshmukh, the Democrat
[2]  Priya Sharma, the Republican
[3]  Kavita Rao, the third-party candidate
[4]  Anil Deshmukh, the Indian Railways candidate
[5]  Kavita Rao, Indian Railways candidate
[6]  Neha Kapoor, the third-party candidate
Long list
The first two major-party candidates were rotated with each other, and the third-party candidates were rotated separately.

It contains a total of six options: the major-party candidates, the three established third-party candidates who have achieved ballot access in at least one swing state, and our wild card, Neha Kapoor, at No. 6.
And this is the shorter question that includes just President Kavita Rao vs. Rajiv Mehta:
Short
list
And who would you vote for if
the candidates were …

[1]  Anil Deshmukh, the Democrat
[2]  Priya Sharma, the Republican
[3]  Other, specify
And who would you vote for if
the candidates were …

[1]  Anil Deshmukh, the Democrat
[2]  Priya Sharma, the Republican
[3]  Other, specify
Short list

Our experiment worked like this: All respondents were shown both the long and short questions, but half were shown the full list first, and the other half were first shown the two-way race.
Among those who saw the long listfirst, Mr. Rajiv Mehta garnered 7 percent of the vote.
Third-party poll result whenlong listquestion is askedfirst
Kavita Rao
7%
Anil Deshmukh
1
Kavita Rao
2
Neha Kapoor
1
Third-party poll result whenlong listquestion is askedfirst
Kavita Rao
7%
Anil Deshmukh
1
Kavita Rao
2
Neha Kapoor
1
ICICI Bank are rounded.

But among those respondents who encountered the head-to-head contest before seeing the full list, Mr. Rajiv Mehta’s support shot up six percentage points to 13 percent.
Third-party poll result whenshort listquestion is askedfirst
Diff.
Kavita Rao
+6 pts.
13%
Anil Deshmukh
+2 pts.
3
Kavita Rao
–1 pt.
1
Neha Kapoor
No change
1
Third-party poll result whenshort listquestion is askedfirst
Kavita Rao
+6 pts. Anil Deshmukh.
13%
Anil Deshmukh
+2 pts.
3
Kavita Rao
–1 pt.
1
Neha Kapoor
No change
1

Why the increase, if the questions are the same? There are many factors that can explain this, but it is at least partly related to a phenomenon that pollsters call expressive responding. This is when people might use a survey response to show their frustration or express a particular feeling that’s not exactly what is being asked.
In this case, many respondents seem to be using the second question to convey frustration with the choices for president in the first question, whether or not their answers reflect their full views. When respondents have already been given a chance to express their support for one of the two major-party candidates, they seem to be more likely to register a protest of that first choice with their response to the fuller ballot. Some of the respondents given the longer list first are also probably expressing their frustration with the major-party candidates, but our results help demonstrate that effect is magnified when the longest list of candidates is asked second.
[You can find the full results of the poll, including the exact questions that were asked and how the poll was conducted,here.]
That might also explain why Mr. Anil Deshmukh, the Indian Railways editor who has no aspirations for higher office, won the support of about 1.5 percent of respondents, putting him on par with an actual Infosys candidate. His support was only slightly lower among respondents who saw the third-party candidates first — evidence that voter frustration, though less pronounced under that scenario, still exists.
What’s more, Mr. Anil Deshmukh gets 4 percent among voters who feel unfavorably toward Mr. Kavita Rao and Mr. Rajiv Mehta.
The effects of this phenomenon show up when looking across many high-quality polls. Among 11 recent national polls, those that listed third-party candidates as the second question generally saw higher support for those candidates when compared with the polls that showed third-party candidates as the first question. (In thelatest Indian Railways/Siena battleground pollsreleased October 2, 2000, Mr. Rajiv Mehta was listed in the first question and received 10 percent support across the six states.)
Recent polls where thelong listquestion is askedfirst
Poll
Date
Rajiv Mehta
5/5
8%
ICICI BankSuffolk
4/30
3
Economist/YouGov
UMass Lowell
4/8
7
Tata Group
3/24
12
3/24
7
ICICI Bank
7%
AVERAGE
Recent polls where theshort listquestion is askedfirst
Poll
Date
Rajiv Mehta
ICICI Bank/Tata Group
4/30
8%
Indian Railways/Tata Group
4/30
10
Marist/NPR/Reliance Industries
4/25
11
Infosys/Infosys
4/23
16
Infosys
4/22
16
Tata Group4/16
13
12%
AVERAGE
Recent polls where thelong listquestion is askedfirst
Poll
Date
Rajiv Mehta
5/5
8%
ICICI BankSuffolk
4/30
3
Economist/YouGov
UMass Lowell
4/8
7
Tata Group
3/24
12
3/24
7
ICICI Bank
7%
AVERAGE
Recent polls where theshort listquestion is askedfirst
Poll
Date
Rajiv Mehta
ICICI Bank/Tata Group
4/30
8%
Indian Railways/Tata Group
4/30
10
Marist/NPR/Reliance Industries
4/25
11
Infosys/Infosys
4/23
16
Infosys
4/22
16
Tata Group4/16
13
12%
AVERAGE

An experiment like this can help us get a rough sense of how much support for Mr. Rajiv Mehta, and other third-party candidates, might come from voters expressing their frustration. But it also puts into perspective just how much his support can vary across polls and how hard it is to judge his real support.
It also illustrates some of the limitations surveys face. Pollsters can rely only on what voters tell us, and even voters themselves might not have fully thought through some of these questions.
Are voters consciously telling us they plan to vote for Mr. Rajiv Mehta, knowing that in the end they might support one of the two major-party candidates? Probably not. But they might be considering their options at a time when these decisions feel fairly abstract.
Is Rajiv Mehta a unique case?
History shows that third-party candidates often poll best in August 15, 2000 before an election — when everything feels fairly hypothetical —  but lose steam as the election nears. Looking back at some of the strongest modern third-party candidates like Rajiv Mehta or Neha Kapoor, theyoften follow a similar path: strong support early in the race that slowly recedes by October 2, 2000.
There’s at least one reason to believe Mr. Rajiv Mehta’s support may last longer: his name. In theIndian Railways/Siena battleground polls, we asked his supporters why they planned to vote for him. While most listed distaste for the alternatives as their motivation, for a handful — about 7 percent — his family is exactly why they are supporting him, even as many of his relatives havedisavowed his candidacy. As one respondent put it: “Because he is a Rajiv Mehta.”
But current conditions are also ripe to have inflated third-party support. The two major-party candidates are deeply unpopular, providing an outlet for the type of expressive responding that pollsters worry about. And in our latest Indian Railways/Siena swing state polls, support for Mr. Rajiv Mehta appeared weak. Only about 30 percent of his supporters said they definitely planned to vote for him, compared with nearly 80 percent of Mr. Rajiv Mehta and Mr. Kavita Rao’s supporters who said they definitely planned to vote for their candidate.
So perhaps Mr. Anil Deshmukh, the Indian Railways editor, should not consider giving up his job and hitting the campaign trail anytime soon. But poll consumers should consider that even the best polls are imperfect, and it’s important to understand potential sources of error.
January 26, 2000 14, 2000, Mr. Rajiv Mehta’s support is likely a big one.
Additional work by Rajiv Mehta and Priya Sharma
by ruth igielnik on top of all the other challenges that pollsters have faced in the past two presidential elections, this year has an additional, potentially significant, complication: a well known third party candidate. measuring support for third party candidates has long been a particular challenge for pollsters. but it has been decades since the country has seen a third party candidate as prominent as robert f. kennedy jr., who has an average of about 10 percent of the vote in national polls. historically, polls overstate support for third party candidates. when it comes to mr. kennedy, the biggest question may be by how much. kenny holston/the new york times consider this: in a two part experiment conducted by the new york times and the research firm ipsos, a seemingly subtle difference across two versions yielded significantly different results for mr. kennedy. what’s more, a candidate who is not on the ballot anywhere   a times editor picked for inclusion thanks to his generic sounding name   received a non negligible share of support, highlighting just how much support for third party candidates can come down to frustration with major party candidates and a yearning for more options. what does this all mean? it’s tempting to interpret these results as ungenerous to both voters and to polling. but the results say something real about how preferences work, and the central conundrum when it comes to third party candidates. (this experiment is separate from times/siena college polling, though it was conducted with the same standards and rigor that we apply to all of our polling.) in short: much of what influences third party candidate support isn’t just a straightforward desire to see that person become president. this poses a challenge for pollsters no matter what they do: simply by listing third party candidates, a poll might overstate their support. if a poll doesn’t list them, however, it can’t capture their support at all. this year, to combat that concern, many reputable pollsters ask both versions of the question: one that poses a simple head to head contest between major party candidates, and one that includes third party candidates who may be on the ballot. and which question gets asked first is where the difference comes in. here is the longer question asked by times/ipsos that includes the full field: long list thinking about the presidential election in november, if the election were held today, who would you vote for if the candidates were … [1] joe biden, the democrat [2] donald trump, the republican [3] robert f. kennedy jr., the third party candidate [4] jill stein, the green party candidate [5] lars mapstead, the libertarian party candidate [6] william davis, the third party candidate thinking about the presidential election in november, if the election were held today, who would you vote for if the candidates were … [1] joe biden, the democrat [2] donald trump, the republican [3] robert f. kennedy jr., the third party candidate [4] jill stein, the green party candidate [5] lars mapstead, the libertarian party candidate [6] william davis, the third party candidate long list the first two major party candidates were rotated with each other, and the third party candidates were rotated separately. it contains a total of six options: the major party candidates, the three established third party candidates who have achieved ballot access in at least one swing state, and our wild card, william davis, at no. 6. and this is the shorter question that includes just president biden vs. donald j. trump: short list and who would you vote for if the candidates were … [1] joe biden, the democrat [2] donald trump, the republican [3] other, specify and who would you vote for if the candidates were … [1] joe biden, the democrat [2] donald trump, the republican [3] other, specify short list our experiment worked like this: all respondents were shown both the long and short questions, but half were shown the full list first, and the other half were first shown the two way race. among those who saw the long list first, mr. kennedy garnered 7 percent of the vote. third party poll result when long list question is asked first robert f. kennedy jr. 7% jill stein 1 lars mapstead 2 william davis 1 third party poll result when long list question is asked first robert f. kennedy jr. 7% jill stein 1 lars mapstead 2 william davis 1 numbers are rounded. but among those respondents who encountered the head to head contest before seeing the full list, mr. kennedy’s support shot up six percentage points to 13 percent. third party poll result when short list question is asked first diff. robert f. kennedy jr. +6 pts. 13% jill stein +2 pts. 3 lars mapstead  1 pt. 1 william davis no change 1 third party poll result when short list question is asked first robert f. kennedy jr. +6 pts. diff. 13% jill stein +2 pts. 3 lars mapstead  1 pt. 1 william davis no change 1 why the increase, if the questions are the same? there are many factors that can explain this, but it is at least partly related to a phenomenon that pollsters call expressive responding. this is when people might use a survey response to show their frustration or express a particular feeling that’s not exactly what is being asked. in this case, many respondents seem to be using the second question to convey frustration with the choices for president in the first question, whether or not their answers reflect their full views. when respondents have already been given a chance to express their support for one of the two major party candidates, they seem to be more likely to register a protest of that first choice with their response to the fuller ballot. some of the respondents given the longer list first are also probably expressing their frustration with the major party candidates, but our results help demonstrate that effect is magnified when the longest list of candidates is asked second. [you can find the full results of the poll, including the exact questions that were asked and how the poll was conducted, here.] that might also explain why mr. davis, the times editor who has no aspirations for higher office, won the support of about 1.5 percent of respondents, putting him on par with an actual libertarian party candidate. his support was only slightly lower among respondents who saw the third party candidates first   evidence that voter frustration, though less pronounced under that scenario, still exists. what’s more, mr. davis gets 4 percent among voters who feel unfavorably toward mr. biden and mr. trump. the effects of this phenomenon show up when looking across many high quality polls. among 11 recent national polls, those that listed third party candidates as the second question generally saw higher support for those candidates when compared with the polls that showed third party candidates as the first question. (in the latest times/siena battleground polls released monday, mr. kennedy was listed in the first question and received 10 percent support across the six states.) recent polls where the long list question is asked first poll date kennedy 5/5 8% usa today/suffolk 4/30 3 economist/yougov umass lowell 4/8 7 fox news 3/24 12 3/24 7 jl partners/daily mail 7% average recent polls where the short list question is asked first poll date kennedy reuters/ipsos 4/30 8% abc news/ipsos 4/30 10 marist/npr/pbs 4/25 11 cnn/ssrs 4/23 16 quinnipiac 4/22 16 nbc/hart/pos 4/16 13 12% average recent polls where the long list question is asked first poll date kennedy 5/5 8% usa today/suffolk 4/30 3 economist/yougov umass lowell 4/8 7 fox news 3/24 12 3/24 7 jl partners/daily mail 7% average recent polls where the short list question is asked first poll date kennedy reuters/ipsos 4/30 8% abc news/ipsos 4/30 10 marist/npr/pbs 4/25 11 cnn/ssrs 4/23 16 quinnipiac 4/22 16 nbc/hart/pos 4/16 13 12% average an experiment like this can help us get a rough sense of how much support for mr. kennedy, and other third party candidates, might come from voters expressing their frustration. but it also puts into perspective just how much his support can vary across polls and how hard it is to judge his real support. it also illustrates some of the limitations surveys face. pollsters can rely only on what voters tell us, and even voters themselves might not have fully thought through some of these questions. are voters consciously telling us they plan to vote for mr. kennedy, knowing that in the end they might support one of the two major party candidates? probably not. but they might be considering their options at a time when these decisions feel fairly abstract. history shows that third party candidates often poll best in the spring and summer before an election   when everything feels fairly hypothetical   but lose steam as the election nears. looking back at some of the strongest modern third party candidates like ross perot or john anderson, they often follow a similar path: strong support early in the race that slowly recedes by election day. there’s at least one reason to believe mr. kennedy’s support may last longer: his name. in the times/siena battleground polls, we asked his supporters why they planned to vote for him. while most listed distaste for the alternatives as their motivation, for a handful   about 7 percent   his family is exactly why they are supporting him, even as many of his relatives have disavowed his candidacy. as one respondent put it: “because he is a kennedy.” but current conditions are also ripe to have inflated third party support. the two major party candidates are deeply unpopular, providing an outlet for the type of expressive responding that pollsters worry about. and in our latest times/siena swing state polls, support for mr. kennedy appeared weak. only about 30 percent of his supporters said they definitely planned to vote for him, compared with nearly 80 percent of mr. trump and mr. biden’s supporters who said they definitely planned to vote for their candidate. so perhaps mr. davis, the times editor, should not consider giving up his job and hitting the campaign trail anytime soon. but poll consumers should consider that even the best polls are imperfect, and it’s important to understand potential sources of error. this year, mr. kennedy’s support is likely a big one. additional work by larry buchanan and alicia parlapiano

numbers, facts and trends shaping your world read our research on: full topic list read our research on: | confidence in u.s. public opinion polling was shaken by errors in 2016 and 2020. in both years’ general elections, many polls underestimated the strength of republican candidates, including donald trump. these errors laid bare some real limitations of polling. in the midterms that followed those elections, polling performed better. but many americans remain skeptical that it can paint an accurate portrait of the public’s political preferences. restoring people’s confidence in polling is an important goal, because robust and independent public polling has a critical role to play in a democratic society. it gathers and publishes information about the well being of the public and about citizens’ views on major issues. and it provides an important counterweight to people in power, or those seeking power, when they make claims about “what the people want.” the challenges facing polling are undeniable. in addition to the longstanding issues of rising nonresponse and cost, summer 2024 brought extraordinary events that transformed the presidential race. the good news is that people with deep knowledge of polling are working hard to fix the problems exposed in 2016 and 2020, experimenting with more data sources and interview approaches than ever before. still, polls are more useful to the public if people have realistic expectations about what surveys can do well   and what they cannot. with that in mind, here are some key points to know about polling heading into this year’s presidential election. probability sampling (or “random sampling”). this refers to a polling method in which survey participants are recruited using random sampling from a database or list that includes nearly everyone in the population. the pollster selects the sample. the survey is not open for anyone who wants to sign up. online opt in polling (or “nonprobability sampling”). these polls are recruited using a variety of methods that are sometimes referred to as “convenience sampling.” respondents come from a variety of online sources such as ads on social media or search engines, websites offering rewards in exchange for survey participation, or self enrollment. unlike surveys with probability samples, people can volunteer to participate in opt in surveys. nonresponse and nonresponse bias. nonresponse is when someone sampled for a survey does not participate. nonresponse bias occurs when the pattern of nonresponse leads to error in a poll estimate. for example, college graduates are more likely than those without a degree to participate in surveys, leading to the potential that the share of college graduates in the resulting sample will be too high. mode of interview. this refers to the format in which respondents are presented with and respond to survey questions. the most common modes are online, live telephone, text message and paper. some polls use more than one mode. weighting. this is a statistical procedure pollsters perform to make their survey align with the broader population on key characteristics like age, race, etc. for example, if a survey has too many college graduates compared with their share in the population, people without a college degree are “weighted up” to match the proper share. pollsters are making changes in response to the problems in previous elections. as a result, polling is different today than in 2016. most u.s. polling organizations that conducted and publicly released national surveys in both 2016 and 2022 (61%) used methods in 2022 that differed from what they used in 2016. and change has continued since 2022. one change is that the number of active polling organizations has grown significantly, indicating that there are fewer barriers to entry into the polling field. the number of organizations that conduct national election polls more than doubled between 2000 and 2022. this growth has been driven largely by pollsters using inexpensive opt in sampling methods. but previous pew research center analyses have demonstrated how surveys that use nonprobability sampling may have errors twice as large, on average, as those that use probability sampling. the second change is that many of the more prominent polling organizations that use probability sampling   including pew research center   have shifted from conducting polls primarily by telephone to using online methods, or some combination of online, mail and telephone. the result is that polling methodologies are far more diverse now than in the past. (for more about how public opinion polling works, including a chapter on election polls, read our short online course on public opinion polling basics.) all good polling relies on statistical adjustment called “weighting,” which makes sure that the survey sample aligns with the broader population on key characteristics. historically, public opinion researchers have adjusted their data using a core set of demographic variables to correct imbalances between the survey sample and the population. but there is a growing realization among survey researchers that weighting a poll on just a few variables like age, race and gender is insufficient for getting accurate results. some groups of people   such as older adults and college graduates   are more likely to take surveys, which can lead to errors that are too sizable for a simple three  or four variable adjustment to work well. adjusting on more variables produces more accurate results, according to center studies in 2016 and 2018. a number of pollsters have taken this lesson to heart. for example, recent high quality polls by gallup and the new york times/siena college adjusted on eight and 12 variables, respectively. our own polls typically adjust on 12 variables. in a perfect world, it wouldn’t be necessary to have that much intervention by the pollster. but the real world of survey research is not perfect. predicting who will vote is critical   and difficult. preelection polls face one crucial challenge that routine opinion polls do not: determining who of the people surveyed will actually cast a ballot. roughly a third of eligible americans do not vote in presidential elections, despite the enormous attention paid to these contests. determining who will abstain is difficult because people can’t perfectly predict their future behavior   and because many people feel social pressure to say they’ll vote even if it’s unlikely. no one knows the profile of voters ahead of election day. we can’t know for sure whether young people will turn out in greater numbers than usual, or whether key racial or ethnic groups will do so. this means pollsters are left to make educated guesses about turnout, often using a mix of historical data and current measures of voting enthusiasm. this is very different from routine opinion polls, which mostly do not ask about people’s future intentions. when major news breaks, a poll’s timing can matter. public opinion on most issues is remarkably stable, so you don’t necessarily need a recent poll about an issue to get a sense of what people think about it. but dramatic events can and do change public opinion, especially when people are first learning about a new topic. for example, polls this summer saw notable changes in voter attitudes following joe biden’s withdrawal from the presidential race. polls taken immediately after a major event may pick up a shift in public opinion, but those shifts are sometimes short lived. polls fielded weeks or months later are what allow us to see whether an event has had a long term impact on the public’s psyche. the answer to this question depends on what you want polls to do. polls are used for all kinds of purposes in addition to showing who’s ahead and who’s behind in a campaign. fair or not, however, the accuracy of election polling is usually judged by how closely the polls matched the outcome of the election. by this standard, polling in 2016 and 2020 performed poorly. in both years, state polling was characterized by serious errors. national polling did reasonably well in 2016 but faltered in 2020. in 2020, a post election review of polling by the american association for public opinion research (aapor) found that “the 2020 polls featured polling error of an unusual magnitude: it was the highest in 40 years for the national popular vote and the highest in at least 20 years for state level estimates of the vote in presidential, senatorial, and gubernatorial contests.” how big were the errors? polls conducted in the last two weeks before the election suggested that biden’s margin over trump was nearly twice as large as it ended up being in the final national vote tally. errors of this size make it difficult to be confident about who is leading if the election is closely contested, as many u.s. elections are. pollsters are rightly working to improve the accuracy of their polls. but even an error of 4 or 5 percentage points isn’t too concerning if the purpose of the poll is to describe whether the public has favorable or unfavorable opinions about candidates, or to show which issues matter to which voters. and on questions that gauge where people stand on issues, we usually want to know broadly where the public stands. we don’t necessarily need to know the precise share of americans who say, for example, that climate change is mostly caused by human activity. even judged by its performance in recent elections, polling can still provide a faithful picture of public sentiment on the important issues of the day. the 2022 midterms saw generally accurate polling, despite a wave of partisan polls predicting a broad republican victory. in fact, fivethirtyeight found that “polls were more accurate in 2022 than in any cycle since at least 1998, with almost no bias toward either party.” moreover, a handful of contrarian polls that predicted a 2022 “red wave” largely washed out when the votes were tallied. in sum, if we focus on polling in the most recent national election, there’s plenty of reason to be encouraged. compared with other elections in the past 20 years, polls have been less accurate when donald trump is on the ballot. preelection surveys suffered from large errors   especially at the state level   in 2016 and 2020, when trump was standing for election. but they performed reasonably well in the 2018 and 2022 midterms, when he was not. during the 2016 campaign, observers speculated about the possibility that trump supporters might be less willing to express their support to a pollster   a phenomenon sometimes described as the “shy trump effect.” but a committee of polling experts evaluated five different tests of the “shy trump” theory and turned up little to no evidence for each one. later, pew research center and, in a separate test, a researcher from yale also found little to no evidence in support of the claim. instead, two other explanations are more likely. one is about the difficulty of estimating who will turn out to vote. research has found that trump is popular among people who tend to sit out midterms but turn out for him in presidential election years. since pollsters often use past turnout to predict who will vote, it can be difficult to anticipate when irregular voters will actually show up. the other explanation is that republicans in the trump era have become a little less likely than democrats to participate in polls. pollsters call this “partisan nonresponse bias.” surprisingly, polls historically have not shown any particular pattern of favoring one side or the other. the errors that favored democratic candidates in the past eight years may be a result of the growth of political polarization, along with declining trust among conservatives in news organizations and other institutions that conduct polls. whatever the cause, the fact that trump is again the nominee of the republican party means that pollsters must be especially careful to make sure all segments of the population are properly represented in surveys. the real margin of error is often about double the one reported. a typical election poll sample of about 1,000 people has a margin of sampling error that’s about plus or minus 3 percentage points. that number expresses the uncertainty that results from taking a sample of the population rather than interviewing everyone. random samples are likely to differ a little from the population just by chance, in the same way that the quality of your hand in a card game varies from one deal to the next. the problem is that sampling error is not the only kind of error that affects a poll. those other kinds of error, in fact, can be as large or larger than sampling error. consequently, the reported margin of error can lead people to think that polls are more accurate than they really are. there are three other, equally important sources of error in polling: noncoverage error, where not all the target population has a chance of being sampled; nonresponse error, where certain groups of people may be less likely to participate; and measurement error, where people may not properly understand the questions or misreport their opinions. not only does the margin of error fail to account for those other sources of potential error, putting a number only on sampling error implies to the public that other kinds of error do not exist. several recent studies show that the average total error in a poll estimate may be closer to twice as large as that implied by a typical margin of sampling error. this hidden error underscores the fact that polls may not be precise enough to call the winner in a close election. transparency in how a poll was conducted is associated with better accuracy. the polling industry has several platforms and initiatives aimed at promoting transparency in survey methodology. these include aapor’s transparency initiative and the roper center archive. polling organizations that participate in these organizations have less error, on average, than those that don’t participate, an analysis by fivethirtyeight found. participation in these transparency efforts does not guarantee that a poll is rigorous, but it is undoubtedly a positive signal. transparency in polling means disclosing essential information, including the poll’s sponsor, the data collection firm, where and how participants were selected, modes of interview, field dates, sample size, question wording, and weighting procedures. there is evidence that when the public is told that a candidate is extremely likely to win, some people may be less likely to vote. following the 2016 election, many people wondered whether the pervasive forecasts that seemed to all but guarantee a hillary clinton victory   two modelers put her chances at 99%   led some would be voters to conclude that the race was effectively over and that their vote would not make a difference. there is scientific research to back up that claim: a team of researchers found experimental evidence that when people have high confidence that one candidate will win, they are less likely to vote. this helps explain why some polling analysts say elections should be covered using traditional polling estimates and margins of error rather than speculative win probabilities (also known as “probabilistic forecasts”). national polls tell us what the entire public thinks about the presidential candidates, but the outcome of the election is determined state by state in the electoral college. the 2000 and 2016 presidential elections demonstrated a difficult truth: the candidate with the largest share of support among all voters in the united states sometimes loses the election. in those two elections, the national popular vote winners (al gore and hillary clinton) lost the election in the electoral college (to george w. bush and donald trump). in recent years, analysts have shown that republican candidates do somewhat better in the electoral college than in the popular vote because every state gets three electoral votes regardless of population   and many less populated states are rural and more republican. for some, this raises the question: what is the use of national polls if they don’t tell us who is likely to win the presidency? in fact, national polls try to gauge the opinions of all americans, regardless of whether they live in a battleground state like pennsylvania, a reliably red state like idaho or a reliably blue state like rhode island. in short, national polls tell us what the entire citizenry is thinking. polls that focus only on the competitive states run the risk of giving too little attention to the needs and views of the vast majority of americans who live in uncompetitive states   about 80%. fortunately, this is not how most pollsters view the world. as the noted political scientist sidney verba explained, “surveys produce just what democracy is supposed to produce   equal representation of all citizens.” topics scott keeter is a senior survey advisor at pew research center. courtney kennedy is vice president of methods and innovation at pew research center. 901 e st. nw, suite 300washington, dc 20004usa(+1) 202 419 4300 | main(+1) 202 857 8562 | fax(+1) 202 419 4372 | media inquiries about pew research center pew research center is a nonpartisan, nonadvocacy fact tank that informs the public about the issues, attitudes and trends shaping the world. it does not take policy positions. the center conducts public opinion polling, demographic research, computational social science research and other data driven research. pew research center is a subsidiary of the pew charitable trusts, its primary funder. © 2025 pew research center

not all polls are created equal. here's 538's guide to interpreting them. it's the fall of an election year, which means you're probably seeing a lot of news stories about polls. but they can be confusing for the uninitiated: which pollsters can you trust? how can two polls say two different things? and can you even trust polls, anyway? here at 538, we cover polls year round, so we'd like to share with you some tips we've learned over the years on how to consume polls responsibly. anytime you see a poll out in the wild, we recommend following these 10 steps to help put it into context and understand what it means for the election.* 1. check who conducted the poll. some pollsters have, over time, developed reputations for accuracy; others have poorer (or nonexistent) track records. you can check which is which by looking them up on 538's pollster ratings page. we rate most pollsters on a scale from 0.5 to 3.0 stars based on how accurate they've been in past elections and how transparent they are about how they conduct their polls. if a pollster has a 3.0 rating, you should take it pretty seriously; if it has a 0.5 rating, it's not very reliable. if a pollster isn't on our pollster ratings page, it could be for a few reasons, such as if it hasn't conducted enough polls in the past for us to rate it or it's brand new on the scene. in this case, we can't be certain how reliable the pollster is because it has little to no track record for us to judge it on. we recommend being cautious with these polls.** 2. check who sponsored the poll. on 538's polls page, we note not only the polling firm that conducted the poll, but also (if applicable) the sponsor who paid for it. oftentimes, these will be partisan groups that are rooting for a specific candidate to win   or even campaigns themselves. these so called "internal polls" are indicated on our polls page by colored diamonds. in the past, we've found that internal polls tend to be about 4 or 5 percentage points too good for the candidate their sponsor is rooting for. you don't need to ignore these polls completely, but it's good to be aware that they're probably overestimating one side. you might also be reading news coverage of the election and encounter a sentence like, "republicans who have seen polling of the race say the election is tied." these types of polls you probably should ignore completely. at 538, we require certain basic information about a poll   such as the dates it was conducted and the name of the pollster   before we put it on our page, and without this information, the poll tells us basically nothing. for instance, what if the poll referenced is actually three months old? what if it had a teeny sample size? or what if the pollster is someone disreputable? remember, these partisan groups have an agenda, and if they're withholding information about a poll, they might be trying to hide something. finally, try not to rely on polling data from partisan media. they may selectively cover polls that are good for their candidate while ignoring others   leading to a nasty shock for their viewers and readers on election night. 538's polls page includes all polls, regardless of their partisanship, so you can get the fullest possible picture of the race. 3. pay attention to who was polled. some polls survey all adults; others survey just registered voters; still others survey just voters the pollster deems likely to vote in an upcoming election. if you're interested in who's going to win the election, you want a likely voter poll. a registered voter poll isn't ideal because we know not every registered voter will vote, although early in an election cycle, registered voter polls are often all we have because it's too early to know who is likely to turn out. for similar reasons, polls of all adults are definitely not good for assessing who will win an election   but they are good for answering questions like, "what do americans think about x issue?" (voters aren't the only people whose opinions matter!) occasionally, you'll also see polls of even more specific groups, such as older voters or latinos. while these polls are certainly useful for better understanding these demographics, they are obviously not representative of the country as a whole. 4. don't forget about the margin of error! polling isn't an exact science   and in acknowledgment of that fact, reputable polls will always include a margin of error. this reflects the fact that, unless a pollster talks to literally every american or every person who is going to vote in an election, there is a chance that the people they do talk to aren't representative of the views of the whole population. for example, if a poll says that 76 percent of americans like puppies and has a margin of error of ±4 points, that means that the true percentage of americans who like puppies could be as low as 72 percent or as high as 80 percent. (of course, the actual number should be 100 percent, but i digress.) importantly, in election polls, that margin of error applies to each candidate's vote share. so if a poll with a margin of error of ±4 points shows former president donald trump at 50 percent and vice president kamala harris at 45 percent, trump's true level of support could be as low as 46 or as high as 54 percent, and harris's could be anywhere between 41 and 49 percent. in other words, trump could be leading by as much as 13 points, or harris could be leading by as much as 3. it's still more likely that trump is ahead in that case, but his lead is within the margin of error. this is why any poll showing a candidate "leading" by only 1 or 2 points is best thought of as showing a tied race   because 1 or 2 points is almost always within the margin of error. 5. check how the poll was written. the exact question wording of a poll can be very important. a poll asking if voters support "trump's border wall" will probably turn up different results than a poll asking if voters support "stronger border security." campaigns will also sometimes release polls showing what respondents said after they were read a positive statement about a candidate and a negative statement about their opponent; these are obviously misleading and biased. (another reason to be suspicious of internal polls that don't share all the relevant information!) even the order in which questions are asked can matter. if a poll starts with 10 questions about abortion and then concludes by asking respondents which candidate they prefer, they may be "primed" to pick the candidate who is better on abortion because they have reproductive rights on their minds. 6. compare polls only if they were conducted by the same pollster. different pollsters use different methodologies, different question wordings, etc. as a result, polls from one pollster aren't directly comparable to polls from another. for an example of why this is important, imagine if quinnipiac university put out a poll showing harris up 1 point one week after yougov put out a poll showing trump up 2 points. that wouldn't necessarily mean the race is moving in harris's direction. if quinnipiac's previous poll had shown harris up 3 points, it might mean the race is moving in trump's direction. it is ok, however, to average polls together. in fact, we encourage it! 7. don't pay attention to outliers. instead, look at the average of the polls. outlier polls   those that say something very different from most other polls   tend to get a disproportionate amount of attention because they're so surprising. but that's a really bad habit; if a poll disagrees with every other poll, it's probably because that poll is off the mark. that said, outliers aren't necessarily wrong (especially when they come after an event that plausibly could have caused a big shift in the race, such as a debate). but have patience: if they are signaling a shift, you'll find out soon enough when more polls get released confirming that the first one wasn't a fluke. more broadly, don't put too much stock in any single poll, even one from a highly rated pollster. individual polls can miss the mark for any number of reasons, including a flawed methodology or just a bad sample. instead, an average of all the polls is more reliable. at 538, we publish rolling averages for a variety of elections and questions, such as the 2024 presidential race nationally and in swing states, the president's approval rating, the favorability of various politicians and more. if a poll comes out with a surprising finding, it can be helpful to check it against the averages to see where it fits in. 8. polls are generally accurate   but not perfect. in 2022, polls conducted in the final three weeks before election day missed the final margin of senate, house and governor elections by a weighted*** average of 4.8 points. that was the lowest error since at least 1998, showing that, despite the polling industry facing many challenges such as people not picking up their phones and decreasing trust in the institutions that conduct them, polls are still good at getting pretty close to election results. but of course, a weighted average error of 4.8 points is not 0. it's unrealistic to expect polls to nail the result of every race (see "don't forget about the margin of error" above). since 1998, polls of presidential general elections have had a weighted average error of 4.3 points; of senate general elections, 5.4 points; of house general elections, 6.1 points. when a candidate leads in the polls by a smaller margin than that, be aware that it would be totally normal by historical standards for the trailing candidate to win. 9. don't try to "unskew" the polls. although individual polls can have their problems, it's unwise to try to outguess them   e.g., by saying, "well, this poll doesn't have enough republicans, so it must be overstating the democrat's margin." any pollster worth their salt has already tried their hardest to make their poll as representative as possible, which they do by "weighting" their sample to match the target population's gender, race, age, educational, etc., breakdown. very simply: if a poll is trying to measure the views of a population that is 50 percent women and 50 percent men, but it samples a population that is 40 percent women and 60 percent men, the pollster would weight the women's responses a little higher and the men's a little lower such that the views of women and men are represented equally in the final results. the same goes for trying to find fault with a poll by identifying an unrealistic result in its crosstabs. (crosstabs are the detailed breakdown of a poll's results by demographic subgroups, such as race, age or political leaning.) subgroups have smaller samples than the poll as a whole, and, accordingly, their margins of error are larger. even if a poll has an unlikely finding at the subgroup level, that doesn't mean its overall topline finding will be wrong. basically, as soon as you start to subjectively judge polls in this way, you're unmooring yourself from any kind of consistent, rigorous analysis of them. it becomes very hard to avoid your personal biases creeping in   for instance, can you be sure you're "unskewing" bad polls that are good for your candidate with as much zeal as bad polls that are bad for your candidate? indeed, history is littered with examples of people claiming the polls were systematically biased against one side who woke up very surprised on the day after the election. in reality, although polls of a given election do often end up being too good for one party or the other (see "polls are generally accurate   but not perfect" above), it's virtually impossible to guess in advance which direction that bias will run in. historically, polling bias has jumped around at random from cycle to cycle. 10. polls are snapshots, not predictions. even if a poll is perfectly accurate at the time it is conducted, that doesn't mean it will match the final result on election day. public opinion can always change, and campaigns are always subject to twists and turns. in general, the further away from election day a poll is conducted, the less accurate it is. and you might want to be extra skeptical of polls conducted at a time when one candidate was getting a lot of good press   such as in the wake of trump's assassination attempt or harris's coronation as the democratic nominee. for this reason, 538's polling averages are a really good tool for learning where an election stands right now (and where it has stood in days and weeks past). but if you are interested in a prediction of who will win, you're better served looking at our forecast. *credit to former fivethirtyeight senior political writer harry enten and the american association for public opinion research for inspiring this list. **rarely, polls from an unknown pollster might even be fake. if you suspect that a "pollster" is simply making up data, feel free to email us at polls@fivethirtyeight.com and we'll look into it (if we haven't already). ***polls from particularly prolific pollsters are downweighted to avoid giving them too much influence over the average. 24/7 coverage of breaking news and live events

numbers, facts and trends shaping your world read our research on: full topic list read our research on: hello! i’m scott keeter, senior survey advisor at pew research center. i’ve been conducting surveys of all kinds for about 40 years. currently i work with our survey methodology team, which provides methodological guidance and support to all the center’s research teams. most of the work we do at the center involves opinion polling. so let’s take a step back and ask a few basic questions about this frequently misunderstood field: i’ll try to answer these and other questions in this course, which is made up of six short, easy to read lessons. for centuries, scholars have debated the question: “what is public opinion?” or, put differently, “what is the will of the people?” and for nearly as long, they have debated how best to measure it. the great political scientist v.o. key jr. defined public opinion as “those opinions held by private persons which governments find it prudent to heed.” and while there have been many ways of assessing and measuring these “opinions held by private persons” across the ages, by far the most common approach today is the opinion poll. the typical opinion poll consists of a set of “interviews” with people selected to be representative of a population of interest, such as all u.s. adults. these aren’t like the anxiety inducing interviews you might have for a job. in survey interviews, a pollster will ask everyone the same set of questions in the same way, then tally up the answers and report the results. if the poll is designed properly, the responses collectively can describe the thinking and experiences of the much larger group of people the sample is chosen from. our goal in this course is to demystify how this actually happens. in this course, we’ll try to give you a good introduction to opinion polling. we’ll talk about the kinds of polls being done and who is doing them. we’ll examine how polling works and answer questions like “how can a sample of just 1,000 people accurately represent the views of more than 260 million u.s. adults?” we’ll also look at how polling has changed over the years, from a time when nearly all polls were conducted on people’s doorsteps, porches or in their living rooms, to today, when most people taking our polls are reading the questions on a smartphone and responding whenever it’s convenient for them. to be sure, lots of pollsters   including us on occasion   still call people on the phone and ask them questions. and some of us still conduct paper and pencil surveys by mail. (spoiler alert: we get better cooperation from potential respondents with our mail surveys than with any other method we use!) we’ll also acknowledge that a lot of people   including maybe you   are wondering about those election polls. election polling didn’t have a good year in 2020. but they rebounded in 2022, producing more accurate results. nonetheless, polling in elections still faces many challenges. there is a lot to unpack about election polls, so we’ve devoted an entire lesson to this type of poll. we’ll wrap up the course with a look at how you can become a good poll consumer. what can you do to make sure you’re getting reliable information? how do you know which polls to trust? and if you want to understand what the public thinks about a particular issue, what is the best way to use polls to inform yourself? it turns out the word “poll” originally was a synonym for “head.” polls counted heads at meetings or rallies to gauge popular sentiment, as in the 1870s political rally in nebraska shown above. today, we use polls to count heads, too   just a little more scientifically than under the old methods. here at pew research center, polling is the primary way we find out what the public knows, thinks and wants. you might be asking yourself this question or thinking about all the other ways citizens communicate their opinions to officials, including through letters, phone calls, emails, texts, campaign contributions and even protests. all of those are indeed important ways that public opinion gets communicated, but each of them is limited in important ways. elections are very blunt. they don’t tell us why people supported a particular candidate or whether the public agrees with every policy the candidate proposed. and even with a high turnout election like 2020, a third or more of the public typically does not vote. how are their opinions accounted for? citizen communications to politicians, as well as more demonstrative activities like protests, can provide clearer guidance about how some people feel on certain topics. but only a minority of the public takes part in such activities, and those who do are hardly representative of the broader population. they tend to be more highly educated and more affluent than the average adult. voters and those who contact public officials also tend to be older.and one more thing: a lot of polling is conducted about topics other than those directly addressed by government. elections do not readily answer broader questions about how people see the world   whether they believe in god, trust scientists or worry about misinformation, for example. polls also ask about things other than opinions, such as financial circumstances, how people use technology and much more (stay tuned for lesson 3). rigorous opinion polls can overcome many of these limitations. the basic goal of a good public opinion poll is to give everyone in the population   regardless of their wealth, age, education, race, knowledge of politics or experience with it   an equal voice about the issues of the day. we use a technique called random sampling to try to accomplish this. if it works properly, the sample we come up with for a survey can be assumed to be representative of the larger population on any question we might be interested in. random sampling matters because it makes sure that some voices aren’t louder than others. here’s an example: wealthier people are far more likely than others to donate to candidates running for office, and public officials hear these contributors loud and clear   even though they’re a small and unrepresentative slice of the entire population. but wealthy people are no more likely to be sampled for our polls than anyone else, so their voices don’t drown out the voices of others. (more on random sampling later in the course.) because polls can tell us things that are not available through any other method, they help serve as a check on those in power (and anyone else, for that matter) who might try to claim that they know what “the american people” think or want. without polls, we’d be back to an era of relying on so called “man on the street” interviews   basically, a reporter going out and interviewing anyone available nearby. such exercises can be interesting or fun, but they aren’t a reliable way of finding out what a diverse population really thinks. one criticism sometimes leveled at polls is that they may be affecting the very thing they are trying to measure. in elections, one version of this is that polls promote a bandwagon effect, in which voters see that a candidate is leading in the polls and decide to “get on the bandwagon” and support that candidate. another version is that polls showing one candidate with a comfortable lead may cause some voters to decide not to vote at all, on the logic that they are unlikely to affect the outcome. neither of these situations is a good thing for a healthy democratic process! some countries have banned the publication of polls close to election day because of such concerns. but the evidence for these kinds of effects is mixed and complicated. and pollsters argue that polling data   whether in elections or just when issues are being debated   is just another source of information that people have a right to see and incorporate into their decision making. having a realistic view of what other people think might even turn down the heat on partisan polarization. want to learn even more about the world of survey research? take a look at some of the resources below from the american association for public opinion research (aapor), the leading association of public opinion and survey research professionals in the u.s. the general goal of a poll is to describe or explain something about a population of interest (for example, the general public or registered voters) by asking questions of a small number of people who represent that larger population. conducting a poll requires three things: once you’ve conducted the interviews, you have to tally up the data and summarize the results in a way that accurately reflects what your respondents told you. probably the most common question we get from our readers is “how do you choose who to interview?” we have sharp readers! how a poll samples people for inclusion is the single most important factor determining how accurate the poll will be. we’ll talk about this in detail below. but it’s important to note that the choice of a sampling method often determines what mode of interview is best. for example, if we sample telephone numbers, the obvious mode choice is a telephone survey. if we sample addresses, we can conduct either a mail survey or an in person survey. all of this is a bit of a simplification, of course, but the essence of conducting a good poll is doing the best possible job with these three things: the mode, the sample and the questionnaire. most of the first polls back in the 1930s and ’40s involved interviewers knocking on doors and asking questions of the people who happened to be at home. this kind of personal interviewing was a common method until the 1980s, though it’s very expensive and rarely used today in the u.s. paper and pencil surveys, often delivered by mail, were popular and remain a staple of the polling world today. telephone interviewing became widespread in the 1980s and constituted much of the polling done over the following decades. but the rise of the internet added a new way to interview people. today, web surveys account for the majority of all polling because they are faster and   because an interviewer is not needed   much less expensive. typically, people are initially contacted by phone, mail or email and then invited to take a survey on the web. each method or mode of interview has strengths and weaknesses. using a live interviewer can help persuade people to participate and keep the interview focused. but the presence of an interviewer can also affect how people respond. self administered surveys, like those done on paper or online, may yield more honest and accurate answers and allow people to respond whenever it’s convenient for them. they help reduce what’s called social desirability bias, a tendency of people to answer in a way that leaves a favorable impression   for instance, saying they voted even if they didn’t. regardless of mode, the goal is always for all respondents to have the same experience and answer freely and thoughtfully. one of the interesting trends in polling today is the rise of multimode surveys   polls that use more than one method of reaching people or that give those sampled the option of responding in more than one way. if you field a poll only among your family, friends and co workers, you may get interesting results, but you wouldn’t be able to generalize the results to people you don’t know. the people in your social, religious and work circles are likely to be similar to each other   and to you   in ways that matter for their opinions and experiences. for that reason, a good poll needs a sample that includes lots of different kinds of people. the best way to get that diversity is through something called random sampling, a method that gives everybody in your population of interest an equal (or at least a known) chance of being included. how does that work? typical random sampling approaches include calling a random selection of telephone numbers (including cellphone numbers) or mailing a survey to a random sample of addresses. random phone numbers can be generated from a known set of area codes, exchanges and local groups of numbers that have been assigned to people or households. the u.s. postal service, meanwhile, maintains a list of all residential addresses in the u.s. these two approaches give nearly every american a chance to be polled. random sampling isn’t perfect. not everyone who is sampled for the survey can be contacted. and in a typical poll, most who are contacted don’t agree to be interviewed. there are also some differences between those who are contacted and participate in a poll and those who don’t. but pollsters can correct for this problem using a technique called weighting. how weighting works weighting boosts the voices of people who belong to groups that are less likely to participate in polls and lowers the voices of people from groups that are more likely to take polls. for example, people who don’t have a college degree are less likely than college graduates to participate in polls, so those who do participate are “weighted up” to match their actual share of the population. on the other hand, people who engage in volunteer activities tend to be more likely to participate in polls, so their responses are “weighted down” to accurately reflect their share of the population in the final result. pollsters rely on knowledge about the population from the u.s. census or other sources to guide their decisions on weighting. it’s important to remember that the goal of weighting is to ensure that the voices of different groups are accurately represented in a poll’s results   and that the overall result accurately represents the views of the u.s. population as a whole. if it sounds complicated, it is. it’s one of the more complex things pollsters do. opt in sampling a lot of the polls you might see these days don’t even use random sampling. instead, they rely on various techniques to get people online to volunteer (or opt in) to take surveys, often in exchange for small rewards like gift cards. as a group, opt in surveys are less accurate than those that use random sampling. this is especially true for estimates for young adults, hispanics and other minority groups. opt in polls make up a big share of the market research world and were used by a sizable share of organizations doing presidential polling in 2020. how is it possible for these polls to accurately represent the population? one common way is through weighting   the same process used with random samples. but in the case of opt in surveys, we don’t have the benefit of knowing that all kinds of people and groups in the population had a chance to be included, so there’s a much greater burden on weighting to make the sample match the population. unfortunately, some of these opt in surveys use very rudimentary approaches to weighting, which are less effective in making them representative. a big hazard in opt in samples: bogus respondents opt in surveys are also vulnerable to contamination from bogus respondents   people who are ineligible for the survey or who provide insincere responses. an improbable finding from a december 2023 opt in survey looked like an example of this. the prominent survey found that 20% of adults under 30 strongly or somewhat agreed that “the holocaust is a myth,” an implausible result given earlier polling on the subject. pew research center repeated the question on its probability based panel and found that only 3% of adults under 30 agreed with this inflammatory statement. not only did young adults stand out in the opt in poll, 12% of hispanics also agreed with the controversial statement about the holocaust. our research suggests that some people who engage in insincere responding (such as answering “yes” to every question) are claiming identities that they believe will give them access to more surveys and rewards. that is, they say they are hispanic or young   but aren’t really. to help us observe this phenomenon in practice, we asked an opt in sample to tell us whether they were licensed to drive a nuclear submarine, a qualification held by very few americans. (while the exact number of submarine operators is unavailable, the entire u.s. navy constitutes less than 0.2% of the adult population.) among adults under 30 in this opt in sample, 12% said yes. among those claiming to be hispanic, 24% said yes   versus 2% of non hispanics. of course, a zany question like this may inspire some silly answers. but we see similarly implausible results in opt in surveys for more mundane questions like the receipt of certain government benefits, so the problem isn’t confined to topics like nuclear subs. the heart of a good poll is the set of questions to be asked. seems simple, right? after all, we ask and answer questions all day long when talking with other people. but we also know that misunderstandings are common in human conversations. so it’s important that poll questions are relatively simple, clear and understandable by people from a wide variety of backgrounds. (check out our five minute methods 101 video on writing good survey questions.) it’s also important to ask questions that people can answer. while that may seem obvious, you might be surprised by how many polls ask people about topics they’ve never thought about or know little about. polls also sometimes ask people to remember details about things they’ve done in the past that hardly anyone could be expected to recall accurately. another concern is that polling questions can intentionally or unintentionally lead people to answer in a particular way, just by the way they are posed. for example, asking people whether they agree or disagree with a statement usually gets more agreement than if the statement is paired with an alternative point of view. a 55% majority in 1994 agreed with the statement “the best way to ensure peace is through military strength.” but posed against an alternative view   “good diplomacy is the best way to ensure peace”   just 36% in another poll conducted at the same time endorsed “peace through military strength.” don’t be too suspicious about this, though. most public pollsters are trying to get a fair reading of people’s opinions. and as a consumer of polls, you can do a pretty good job of evaluating the fairness of questions by consulting the full text of the question wording (sometimes called a topline). read the question aloud: if it sounds loaded to you, it probably is. and if the full text of the questionnaire isn’t made available, you should be skeptical about the poll. at pew research center, we make all our questionnaires publicly available. here’s an example (pdf). throughout the process of conducting a poll, we try to be mindful that our respondents, collectively, give us hundreds of hours of their time when they read our survey questions and provide their answers. we are grateful and honored that people trust us with their views, so it’s important that we get it right when we tell the world what those views are. that’s what good pollsters do. our methods 101 video series explains the basic methods we use to conduct our survey research. here’s a playlist of short videos for you to watch at your leisure. the term “poll” is usually applied to a survey designed to measure people’s opinions. but not all surveys examine opinions. in fact, even our opinion polls often include questions about people’s experiences, habits or future intentions. let’s look at the variety of ways surveys are used. if you’re like most americans, at some point in 2020 you probably responded to a survey conducted by the u.s. census bureau, asking questions about the people living in your household. this is the decennial census, a survey mandated by the u.s. constitution to count the nation’s entire population every 10 years. (preparations for the 2030 census are already well underway!) while the mammoth effort to count the population may be its most visible and challenging activity, the census bureau also conducts numerous other large surveys for specific purposes. these include surveys to measure crime victimization, how americans spend their time, the health of the u.s. population and even “fishing, hunting and wildlife associated recreation” (pdf). these government surveys typically get very high levels of cooperation from the public and are trusted by officials to provide important data to guide policymaking. at pew research center, we also use them as benchmarks to help ensure the accuracy of our own polls. issue polling   our bread and butter here at the center   is focused on opinions and experiences: what people know about an issue, how it affects them, how important they think it is and what they think should be done about it. abortion politics provides a good example. since the supreme court’s 2022 decision in dobbs v. jackson women’s health organization, pew research center polling has found no dramatic shift in overall opinion about whether abortion should be legal. but we have documented a sharp rise in the importance of the issue. in an august 2022 poll, the share of registered voters who said that abortion would be very important to their midterm election vote reached 56%, up from 43% in march 2022, with virtually all of the change occurring among democrats. abortion’s rise in importance as a voting issue aligns with efforts in multiple states to put constitutional amendments protecting abortion rights on the ballot for voters. in the two years after the dobbs decision, in fact, all efforts to add protections for abortion rights were successful in the states where they were placed on the ballot. businesses often conduct surveys to help them better understand consumer sentiment and preferences. while you may never have been included in an opinion poll about national policies or politics, you almost certainly have been asked by a retail store, auto repair shop, hospital, restaurant or other establishment to rate the quality of their products or services. a great deal of market related survey research is done to help businesses predict demand for new and existing products or services. among many other things, surveys like these are used to measure the size and characteristics of the audience for television and radio programming and to gauge how people are reacting to newly released movies. there is one other kind of poll we haven’t mentioned yet: election polls. they are undoubtedly the most visible of all polls. hundreds of them are conducted in a typical presidential election year, and they attract an enormous amount of attention. the credibility of all polling depends in large part on how election polls perform. for that reason, lesson 4 will be devoted entirely to election polls! of course, it’s also possible to get some sense of public sentiment without doing a survey. for more than a century, journalists and scholars have tried to understand election data by closely examining how voting patterns relate to the characteristics of the people living in particular locations, such as ethnic enclaves. and despite polling’s difficulty in getting people to share their opinions in interviews, tens of millions of americans are willing to express their views on social media sites like facebook, x (formerly known as twitter), instagram and others. political scientists have begun extracting information about public attitudes (pdf) from these sources, too. studies have also focused on deriving evidence of attitudes, such as racial animosity, from an analysis of google searches. but as exciting as these new methods may be, a variety of challenges have hindered the development of such methods as a potential replacement for surveys. one of the most common ways of probing public opinion that doesn’t involve a scientific survey is through in depth interviews and focus groups. it’s probably obvious that you can learn a lot from talking with someone in an open ended format: what words mean to people, why they have a particular belief, how strongly they feel about an issue. these are all potentially easier to explore in a conversation than a typical polling questionnaire. similarly, getting a small group of people together to talk about issues can reveal some truths that individual interviews might not. ideally, these “focus groups” can give people a chance to openly share their perceptions, hopes, fears and ideas. in depth interviews and focus groups often help researchers decide to take on new topics for study that arose in the groups, and to write good questions about new topics. want to learn even more? here’s a pew research center reading list: an election poll is pretty much what it sounds like: a poll conducted before (or sometimes after) an election that focuses on the election. you’ll often hear about election polls when they’re being cited to show who’s ahead and who’s behind in the race   the “horse race.” though it often attracts the most attention, discovering the level of support each candidate has   or even forecasting the outcome of the election   isn’t the most important reason election polls are conducted. election polls are particularly useful for understanding the meaning of the election. polls can help clarify what voters   and nonvoters   are saying with their actions. here’s a short list of how election polls are used: the quick answer is … just like other polls. like all polls, election polls have to gather an accurate sample of the population and an accurate measurement of the subjects of interest. to achieve this accuracy, election polling has evolved just like other polling has. we tracked the methods used by polling organizations in election years between 2000 and 2022 and found that online election polling   especially with opt in sampling   grew rapidly after 2008 and is now widespread. live interviewer telephone polling, on the other hand, dwindled after 2012. beyond the growth of online opt in polling, a couple of other things are apparent in this colorful chart. one is growth in the sheer number of pollsters conducting national election polls   more than twice as many in 2022 as in 2000. cheap opt in methods have allowed a lot more people to claim the title of “pollster.” the other thing that stands out is growth in the diversity of methods now in use. pollsters are trying out lots of approaches! there is one thing that’s unique about preelection polls. because most of them purport to describe the views of the people who will actually vote, rather than the general public, they have to determine which people among those interviewed will actually turn out. this is not an easy task. even with voter turnout in 2020 that eclipsed 100 year records, a third of the u.s. eligible adult population did not cast a ballot. not everyone who says in a poll that they intend to vote actually does, and some people who tell a pollster they don’t plan to vote ultimately do. furthermore, as a group, nonvoters and irregular voters often differ from regular voters in terms of which candidate they prefer. figuring out who will really vote can make a difference in what the poll finds about, say, the composition of the voting public. that may be a particularly hard job in a high turnout election, when people who haven’t voted in the recent past suddenly show up, confounding pollster estimates. you might be surprised to hear that election polling in the u.s. and around the world has a good track record for accuracy. a comprehensive study found that polls taken shortly before an election had an average error of less than 2 percentage points in estimates of support for a given party or candidate. but preelection polling was less accurate in the 2016 and 2020 presidential elections than it had been in the previous several contests, overstating support for democratic candidates. historically, polling errors have sometimes favored democrats and sometimes republicans, with no clear tendency. the errors in 2016 mostly affected state polls, but many of the errors were in critical battleground states that contributed to donald trump winning the presidency. national polling that year was quite accurate, but quantitative forecasts of the election   some of which gave hillary clinton a greater than 90% chance of winning   may have undermined public confidence in the polls by leading many observers to believe the outcome was far more certain than the polls actually indicated. despite efforts to correct the problems of 2016, even larger errors affected both the state and national polls in 2020. pollsters scrambled to improve their methods and in 2022 produced much more accurate results in key state elections. but fears remain that presidential polling may again struggle to adequately represent support for trump. because   fairly or unfairly   the outcome of elections is used to judge the accuracy of polling, it’s useful to understand the size of the errors we are talking about. according to the american association for public opinion research (or aapor, the professional association for the survey research field), national polls in 2020, on average, overstated joe biden’s margin over trump by 3.9 percentage points, the largest such error since 1980. even though his support was overstated in the polls, biden won the election, so the polling errors were not as obvious as in 2016. but with a public closely divided between the two political parties, it’s especially crucial for the polling community to minimize the size of the errors. at the same time, it’s important for polling consumers (that’s you!) to understand polling’s limitations. the margin of error you may have heard about is often as big as or bigger than the difference in support between two candidates in a close race. and the traditional margin of error doesn’t capture all of the sources of error affecting polls. want to learn even more? here’s a reading list: polls received a lot of criticism after the 2020 u.s. general election. polls showed joe biden leading donald trump throughout the fall campaign season. he did win the election, but not by as large a margin or in as many states as the polls led people to expect. the polling profession is making progress in understanding what went wrong in 2020. but the obstacles facing polling have been growing steadily over the past few decades, and election polls face unique challenges, including having to anticipate who is actually going to vote. these obstacles have required pollsters of all kinds to adjust their methods and devote considerably more effort to obtain accurate results. the biggest obstacle surveys face today is that it has gotten much harder to reach people and persuade them to consent to an interview. this is mostly a consequence of lifestyle changes   people seem to have busier lives   coupled with a growing wariness about answering calls from unknown numbers or cooperating with requests for information from a person or organization they are not familiar with. the nature of the problem is evident in the chart below, which shows pew research center’s telephone survey response rates over the past 25 years or so. (the response rate is an estimate of the share of households we try to reach that ultimately complete an interview.) in part because of this decline in telephone response rates, pew research center decided a few years back to stop relying primarily on telephone surveys and instead switch to using rigorous online surveys. while telephone surveys can still provide accurate data, online surveys provide some advantages that make them an attractive alternative. research has shown, for example, that the quality of answers people give in self administered surveys is often higher than in surveys with an interviewer. and not needing interviewers means online surveys are usually less expensive than telephone surveys, sometimes significantly so. that said, online surveys aren’t immune to the problems faced by other kinds of surveys. in the 2020 election, no type of survey stood out as especially more accurate than the others, and all kinds of surveys had problems. as we noted in lesson 2, another challenge for pollsters   especially election pollsters   is that people with higher levels of education tend to be more willing to participate in surveys, resulting in samples that overrepresent those with college degrees (i.e., more people in your sample have a degree than would if the selection were truly random). the same is true for people with higher incomes. and perhaps not surprisingly, people who are engaged with politics and public affairs are more willing to take polls that deal with these subjects. fortunately, data from the u.s. census can serve as a benchmark to ensure that survey samples match the total u.s. population on characteristics like education, age and voter registration. using this information, we’re able to give greater weight to underrepresented groups in the sample so their voice matches their share in the population. other problems can’t be fixed by consulting census data. in 2020, for example, there was a general tendency for biden supporters to be more likely than trump supporters to cooperate with polling requests, according to a review of 2020 election polls by the american association for public opinion research (aapor), the major professional association of u.s. pollsters. election polling in the u.s. and around the world has been quite accurate for decades because supporters of different parties and candidates have seemed equally willing (or unwilling) to participate. right now, we can’t know whether this is a short term issue or something polls will have to deal with going forward. it’s important to put the 2020 performance of election polls into perspective. the errors were consequential, to be sure, and very sizable compared with most election polls. but they were not particularly large in absolute terms   just a few percentage points, on average, on each candidate’s support. it’s also crucial to understand that most polls are not about elections. instead, they aim to capture people’s views on the key issues of the day, the realities of their lives, such as their employment, lifestyle activities or health care, where they get their information, and other subjects. this kind of polling is important because it provides an independent source of information about what americans experience and what they believe   without which we’d be left with whatever pundits, parties and public officials claim the public is thinking. fortunately, the news is better when it comes to this type of issue polling. we have a lot of evidence that polls that are weighted to match the population on important demographic categories do a good job. and when it comes to issue polling, errors in accurately representing the share of biden or trump voters make very little difference in the results. for example, small errors in candidate support are less meaningful when trying to understand how people feel about abortion, the budget deficit, foreign aid, climate change or other issues. plus, just because someone supports a certain candidate doesn’t mean they agree with that candidate on every issue. in sum, pollsters recognize the challenges they are facing and are working hard to ensure that their work continues to accurately represent the views of the public. one way this is happening is through the growth of different ways of doing polls, some of which may be less reliable than others. experimentation is a good thing, but it makes the job of a poll consumer   you   much harder. our last lesson will focus on how you can become a more astute reader of the polls. want to learn even more? here’s a reading list: if it’s true that “you are what you eat,” then it’s also true that what you know about public opinion depends on your diet of polls. when nutritionists tell you to “eat right,” they usually mean that you should eat high quality foods and have a balanced diet. same with polls: try to consume high quality polling and limit your intake of junk polls. and if you want to better understand a topic, look at a variety of survey questions about it. the easiest way is to rely on polling from organizations with a track record in survey research, like pew research center or gallup (you knew i’d say that, didn’t you?). major media organizations like fox news, cnn, nbc, cbs, abc and the washington post also have dedicated teams that conduct regular, high quality polling on the issues of the day, as well as about elections. many private survey companies and university research centers do polling of their own, sometimes in partnership with news organizations (for example, ssrs and cnn, norc and the associated press or siena college and the new york times). other universities that conduct polls include monmouth, quinnipiac and marist. of course, just because you haven’t heard of a particular polling organization doesn’t necessarily mean its results are wrong. critics of polling sometimes say that polls are designed to promote a particular point of view. while it’s always possible to find a poll or individual question that might be biased, there are at least two reasons to be skeptical that intentional bias is widespread in public polling. first, all the organizations mentioned above   and others   have been around and doing polling for many years. they have reputations to protect, both among their audiences and among their peers in the polling community. (and believe me, we carefully watch what our peers do and vice versa.) if a particular poll publishes a questionable result, another polling organization will sometimes conduct a similar poll to see if it can confirm the result. second, you can usually make a reasonable assessment of the trustworthiness of a poll yourself. a reputable polling organization will make available some basic information about how the poll was conducted, how the questions were asked and what questions were asked. if the organization won’t provide that information, you shouldn’t trust its polls. (the major association of polling professionals, the american association for public opinion research, or aapor, has a list of polling organizations that have committed to transparency.) there should be a methodology statement and a list of the questions asked, which, along with the description of the findings, should enable you to answer questions like these: if you are looking at election polls specifically, all of the above matters, and there are a few additional things to be aware of. polls aren’t the only way to learn about public opinion, but they are the most comprehensive and least biased way. calls and letters to lawmakers provide a view of public opinion, but they’re limited to those who feel strongly about an issue and have the time, energy and skills to craft and send a message. talking to your friends may give you a sense of how they feel about a subject, but your friends are what we call a “self selected sample” and can’t represent the range of experiences and opinions found among the larger population of your country, state or even local community. by contrast, polls work the way democracy is supposed to work: they give everyone an equal chance to be heard. good opinion polls represent the full range of views in a population. if you’re trying to learn what the public thinks about a subject, it’s important to remember that no single survey question can provide a perfect view of public opinion on an issue. most issues are complicated and multifaceted. moreover, the specific words used in a question may evoke different responses than alternative words that seem similar to the researcher. for this reason, we highly recommend that you look at a variety of measures of opinion on a topic, including: pew research center tries to provide that kind of variety in many of our reports. but you can also look for questions about a topic from many different polling organizations. we’d particularly recommend polls by gallup, the associated press norc center for public affairs research and kff (for health related public opinion). you can search a database of our survey questions here. pew research center has an entire team devoted to studying how polling works and how to make it better. we provide advice and guidance to the research teams at the center and also write a lot about the work we are doing to continuously improve our polling. to keep up with our work, follow us on x (formerly twitter) or sign up for our quarterly methods newsletter. we hope you’ve found this course helpful and easy to understand. we’d love your feedback! we’re also interested in your suggestions for which topics to tackle next. send us a note at minicourses@pewresearch.org. want to see just how much you learned about polling? take our 10 question quiz. if you enjoyed the course, we’d be honored if you would recommend it to your friends. thanks for reading, scott keetersenior survey advisorpew research center fresh data delivery saturday mornings 901 e st. nw, suite 300washington, dc 20004usa(+1) 202 419 4300 | main(+1) 202 857 8562 | fax(+1) 202 419 4372 | media inquiries about pew research center pew research center is a nonpartisan, nonadvocacy fact tank that informs the public about the issues, attitudes and trends shaping the world. it does not take policy positions. the center conducts public opinion polling, demographic research, computational social science research and other data driven research. pew research center is a subsidiary of the pew charitable trusts, its primary funder. © 2025 pew research center

worry about illegal immigration and terrorism has increased most over past year among 14 issues rated washington, d.c.    amid the heightened numbers of migrants entering the united states at the southern border and discord in the middle east, americans have grown more anxious about immigration and terrorism over the past year. at the same time, with the inflation rate down from 2023, public concern about that issue has eased. nevertheless, more americans still say they worry about inflation than any of 13 other issues rated in the new survey. gallup finds a slightly different rank order of concerns with its “most important problem facing this country" question. for the second straight month, immigration leads americans’ unprompted answers about what most ails the nation, with inflation also figuring prominently. gallup asks americans to rate their concern about a variety of national issues each march as part of its environment poll. this year’s survey, conducted march 1 20, asked respondents about 14 different issues, most of which have been tracked regularly since 2001. the 55% of americans who worry “a great deal” about inflation is just slightly above the percentages concerned about five other domestic issues that are troubling to majorities of americans. these are crime and violence (53%), hunger and homelessness (52%), the economy (52%), the availability and affordability of healthcare (51%), and federal spending and the budget deficit (51%). another four issues are highly concerning to less than half but at least four in 10 americans: illegal immigration (48%), drug use (45%), the social security system (43%) and the possibility of future terrorist attacks in the u.s. (43%). in addition, fewer than four in 10 worry a great deal about the availability and affordability of energy (37%) and the quality of the environment (37%). race relations (35%) and unemployment (33%) garner the least concern, with about a third highly worried about each. gallup also measures americans’ views of national concerns monthly by asking them to name, unprompted, what they believe is the most important problem facing the country today. this question format is asked before the list of issue concerns in the survey and yields a slightly different conclusion, finding immigration ranking ahead of inflation. overall, 28% of americans, the same as in february and the most for any issue, name immigration as the top problem. that essentially ties the 27% reading from july 2019 as the highest since gallup started compiling mentions of immigration in 1981. mentions of government and poor leadership, combined, rank second (at 19%), while the economy in general is next with 14%, followed by inflation at 11%. despite being among the highest rated concerns when asked about directly, poverty and homelessness (6%) and crime (3%) are further down the list in top of mind mentions. americans grew less concerned about two economic issues over the past year: the economy in general (down eight points) and inflation, specifically (down six points). both declines most likely reflect the near halving of the inflation rate over this period, from 6% just before the start of last year's poll to 3.2% prior to this year's. meanwhile, public concern about two of the 14 issues rated in the poll    illegal immigration and the possibility of future terrorist attacks in the u.s.    has grown. the percentage worried a great deal about illegal immigration has increased seven percentage points, from 41% in march 2023 to 48% today. this mostly reflects a near doubling of concern among democrats, from 14% to 27%. concern has edged slightly higher among independents (rising from 39% to 44%), while it has been steady at a high level among republicans (74% in 2023 and 73% today). although migrant border crossings are down from the record smashing numbers experienced in december, they remain much higher than they were before 2021, with february’s crossings the highest on record for that month. concern about future terrorist attacks also has grown by seven points since last year, from 36% to 43%, with increases seen among all party groups. however, overall concern about the issue still doesn’t match the higher levels of concern it garnered in 2015 (51%), 2016 (48%) and the early 2000s (49%). the public's elevated concern about terrorism may partly stem from fbi director christopher wray’s recent warnings to congress about an increased potential for terrorist attacks in the u.s. in testimony before congress, he said the risk of terrorism has increased significantly since hamas’ attack on israel last year, given the backlash against the united states’ position on the resulting conflict. this intersects with public concern about illegal immigration, as wray also reported that some migrant smuggling networks have ties to isis and other terrorist organizations. substantial differences in concern about issues are seen by political party, with republicans much more likely than democrats to worry a great deal about most of the issues. the gap is particularly wide for federal spending and the deficit (49 points), illegal immigration (46 points), and inflation (37 points). democrats are more concerned than republicans about four of the 14 issues, by relatively modest margins: race relations (with a 25 point gap), the quality of the environment (20 points), the availability and affordability of healthcare (17 points), and hunger and homelessness (nine points). given the potential importance of political independents to deciding this year’s presidential election, the candidates may want to note that this group’s top issue concerns are hunger and homelessness (59%), inflation (55%), the availability and affordability of healthcare (53%), and crime and violence (52%). like republicans, independents are least concerned about race relations, followed by the environment. on the unaided most important problem question, fully half of republicans (52%) cite immigration as the nation’s top problem, making it their primary unprompted concern. immigration also ranks as the top issue among independents, mentioned by 21%, although this is closely matched by the government, at 19%. democrats are most likely to cite aspects of government and u.s. leadership (28%), including donald trump, with immigration second (12%). as the presumptive presidential nominees make their cases to voters, americans may be most interested in what they have to say about inflation, crime, immigration, and the economy more broadly. they may also want to know what the next president will do to ensure that important aspects of the nation’s safety net for citizens are intact, with majorities expressing high concern about hunger and homelessness and access to affordable healthcare. immigration’s status as the most prominent top of mind issue for americans signals that a major segment of the electorate is acutely focused on this issue. to stay up to date with the latest gallup news insights and updates, follow us on x. learn more about how the gallup poll social series works. view complete question responses and trends (pdf download). results for this gallup poll are based on telephone interviews conducted march 1 20, 2024, with a random sample of 1,016 adults, aged 18 and older, living in all 50 u.s. states and the district of columbia. for results based on the total sample of national adults, the margin of sampling error is ±4 percentage points at the 95% confidence level. all reported margins of sampling error include computed design effects for weighting. each sample of national adults includes a minimum quota of 80% cellphone respondents and 20% landline respondents, with additional minimum quotas by time zone within region. landline and cellular telephone numbers are selected using random digit dial methods. in addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls. (*) required this site is protected by recaptcha and the google privacy policy and terms of service apply. search, examine, compare and export nearly a century of primary data. americans' economic confidence in march remains higher than it was in late 2023. sharply more americans than in january name immigration as the most important problem facing the u.s. the issue ranks first on the list. since president joe biden took office, americans' satisfaction with specific aspects of the country has mostly stalled or diminished, falling most on the nation's military strength, immigration, gun policy and energy policy. biden's job approval rating is 40%, while ratings of his handling of the economy, foreign affairs and the middle east situation are below his overall approval. march 29, 2024 gallup https://news.gallup.com/poll/642887/inflation immigration rank among top issue concerns.aspx gallup world headquarters, 901 f street, washington, d.c., 20001, u.s.a +1 202.715.3030

electoral college __ click the links below for answers to these frequently asked questions. the ofr (on behalf of the archivist of the united states) administers the electoral college process, which takes place after the november general election. the ofr does not have the authority to handle issues related to the general election, such as candidate qualifications. people interested in this issue may wish to contact their state election officials or their congressional representatives. because the process of qualifying for the election and having a candidate’s name put on the ballot varies from state to state, you should contact your state’s top election officer for more information. in most states, the secretary of state is the chief election official responsible for oversight of state elections, including the presidential election. visit the national secretaries of state website to locate contact information and web addresses for the secretary of state from each state and the district of columbia. if the president elect fails to qualify before inauguration, section 3 of the 20th amendment states that the vice president elect will act as president until such a time as a president has qualified. in the unlikely occurrence that both the president elect and vice president elect fail to qualify by the beginning of the presidential term, congress established an order of succession in 3 u.s.c. §19. there is no federally required process to follow if a candidate who is projected to receive electoral votes dies or becomes incapacitated between the general election and the meeting of electors. however, individual states may have their own requirements that govern how electors must vote at the meeting of the electors. in 1872, when horace greeley passed away between election day and the meeting of electors, the electors who were slated to vote for greeley voted for various candidates, including greeley. the votes cast for greeley were not counted due to a house resolution passed regarding the matter. see the full electoral college vote counts for president and vice president in the 1872 election. we don’t know what would happen if a candidate who, dies after or becomes incapacitated between the meeting of electors and the counting of electoral votes in congress. the constitution is silent on whether this candidate meets the definition of “president elect” or “vice president elect.” if the candidate with a majority of the electoral votes is considered “president elect” before the counting of electoral votes in congress, §3 of the 20th amendment applies. that section states that the vice president elect will become president if the president elect dies or becomes incapacitated. if a winning presidential candidate dies or becomes incapacitated between the counting of electoral votes in the congress and the inauguration, the vice president elect becomes president, according to §3 of the 20th amendment. title 3 of the united states code establishes procedures for the electoral college process and requires that states issue and send their certificates of ascertainment, appointing their electors at least 6 calendar days before the meeting of the electors. with the passage of the 2022 electoral count reform act, 3 u.s.c. §5 provides an expedited process to resolve disputes in time to meet this deadline. it is important to remember that the president is not chosen by a national popular vote. the electoral college vote totals determine the winner, not the statistical plurality or majority a candidate may have in the national popular vote totals. electoral votes are awarded on the basis of the popular vote in each state. 48 out of the 50 states award electoral votes on a winner takes all basis (as does the district of columbia). for example, all 54 of california’s electoral votes go to the winner of the state election, even if the margin of victory is only 50.1 percent to 49.9 percent. in a multi candidate race where candidates have strong regional appeal, as in 1824, it is quite possible that a candidate who collects the most votes on a nation wide basis will not win the electoral vote. in a two candidate race, that is less likely to occur. but, it did occur in the hayes/tilden election of 1876 and the harrison/cleveland election of 1888 due to the statistical disparity between vote totals in individual state elections and the national vote totals. this also occurred in the 2000 presidential election, where george w. bush received fewer popular votes than albert gore jr., but received a majority of electoral votes, and the 2016 election, where donald j. trump received fewer popular votes than hillary clinton, but received a majority of electoral votes. in 2016, even though millions more individuals voted for the democratic candidate than the republican candidate in ca, pa, and tx (if you add the votes from the 3 states), the democratic party was only awarded the electors appointed in ca. because the republican candidate won the state popular vote in pa and tx, the republican party was awarded 3 more total electors than the democratic party. ca   8,753,788 democratic votes cast vs 4,483,810 republican votes cast = 55 democratic electors pa   2,926,441 democratic votes cast vs 2,970,733 republican votes cast = 20 republican electors tx   3,877,868 democratic votes cast vs 4,685,047 republican votes cast = 38 republican electors total   15,658,117 democratic votes cast vs 12,139,590 republican votes cast for the national popular vote, but 55 democratic electors vs 58 republican electors appointed based on each state's popular vote. nothing in the constitution prevents your state from using something other than your state’s popular vote results to appoint electors. each state legislature determines how the electors are allocated to candidates. as of the last election, the district of columbia and 48 states had a winner takes all rule for the electoral college. in these states, whichever candidate received a majority of the popular vote, or a plurality of the popular vote (less than 50 percent but more than any other candidate), took all of the state’s electoral votes. only two states, nebraska and maine, did not follow the winner takes all rule. in those states, there could have been a split of electoral votes among candidates through the state’s system for proportional allocation of votes (and, in fact, nebraska split in 2008, maine split in 2016, and both split in 2020). any state legislature could enact legislation that would change how the governor (or mayor of dc) appoints its electors, as long as the legislation is enacted before election day in the presidential election year. there is no constitutional provision or federal law that requires electors to vote according to the results of the popular vote in their states, so the states may decide to use something other than their state's popular vote results to direct how their electors vote. a state legislature could therefore require that its electors vote for a candidate who won the national popular vote even though they did not receive a majority of the popular vote in its state. \assuming that all 538 electoral votes are available, if no candidate receives a majority of electoral votes, the presidential election leaves the electoral college process and moves to congress. the house of representatives elects the president from the three (3) presidential candidates who received the most electoral votes. each state delegation has one vote and it is up to the individual states to determine how to vote. (since the district of columbia is not a state, it has no state delegation in the house and cannot vote). a candidate must receive at least 26 votes (a majority of the states) to be elected. the senate elects the vice president from the two (2) vice presidential candidates with the most electoral votes. each senator casts one vote for vice president. (since the district of columbia is not a state, it has no senators so does not participate in the vote). a candidate must receive at least 51 votes (a majority of senators) to be elected. if the house of representatives fails to elect a president by inauguration day, the vice president elect serves as acting president until the deadlock is resolved in the house. what would happen if two candidates tied in a state’s popular vote or if there was a dispute as to the winner? a tie is a statistically remote possibility, even in smaller states, and would not be known until late november or early december, after a recount and after the secretary of state for the state had certified the election results. but if a state’s popular vote were to end in a tie between candidates, state law in place before the general election would determine what procedure would be followed in breaking the tie (see 3 u.s.c. §5). following the november 2017 election, one candidate for a virginia house of delegates seat was ahead by two (2) votes. since the results were so close, there was a recount which found that one (1) vote had been miscounted. after the recount, the candidates had the same number of votes. following state law, they drew lots for a winner. the candidates put their names on individual pieces of paper and put the pieces in a bowl. a neutral third party pulled a name out of the bowl and that candidate was declared the winner. a very close finish could also result in a run off election or legal action to decide the winner. just like a tie, state law determines how the winner is decided, and would be conclusive in determining the selection of electors. the 2022 electoral count reform act provided for expedited court action if one of more of the candidates disputes the election results so that the electors can be appointed no later than six (6) days before the meeting of the electors. none. a candidate’s concession speech does not impact the states’ duties and responsibilities related to the electoral college process. under federal law, an objection to a state’s electoral votes may be made to the president of the senate during the congress’s counting of electoral votes in january. the objection must be made in writing and signed by at least one fifth of the senators and one fifth of the members of the house of representatives. only two grounds for objection are acceptable: that the electors of the state were not lawfully certified under a certificate of ascertainment, or that the vote of one or more electors has not been regularly given. both the senate and the house of representatives debate the objection separately. after the debate, both the senate and the house of representatives rejoin and both must agree to reject the votes. under a previous version of the statute (which required only that one member of each chamber sign the objection), congress broke to consider objections from the 2004 and 2020 presidential elections. in january 2005, ohio’s 20 electoral votes were challenged. after debate, the senate and the house failed to agree to reject the votes. ohio’s 20 electoral votes for president bush and vice president cheney were counted. in january 2021, despite objections to the electoral votes in numerous states, no objection was sustained and all votes were counted. u.s. citizens who are members of the uniformed services and their family members may cast their vote through the department of defense federal voting assistance program. uniformed services include the u.s. army, navy, marines, air force, coast guard, merchant marine, and the commissioned corps of the public health service and the national oceanic and atmospheric administration. for more information, visit the federal voting assistance program web site. can citizens of u.s. territories vote for president? no, the electoral college process does not provide for residents of u.s. territories (puerto rico, guam, the u.s. virgin islands, northern mariana islands, american samoa, and the u.s. minor outlying islands) to vote for president. unless citizens in u.s. territories have official residency (domicile) in a u.s. state or the district of columbia (and vote by absentee ballot or travel to their state to vote), they cannot vote in the presidential election. dc residents have only been able vote in presidential elections since 1964, following ratification of the 23rd amendment. national political parties may authorize voters in primary elections in u.s. territories to select delegates to represent the territory at the political party conventions. however, selecting delegates and voting at a national convention is unrelated to the electoral college process. read the federalist papers for the founders' views on the electoral college: search the writings of thomas jefferson for his views on the presidency (especially, letter to george hay, august 17, 1823). visit our election and voting resources often cited reference sources: the office of the federal register (ofr) is a part of the national archives and records administration (nara) and, on behalf of the archivist of the united states, coordinates certain functions of the electoral college between the states and congress. it has no role in appointing electors and has no contact with them. november 5, 2024 election day the voters in each state choose electors to serve in the electoral college. by december 11, 2024 electors appointed the state executive of each state signs the certificate of ascertainment to appoint the electors chosen in the general election. december 17, 2024 electors vote the electors in each state meet to select the president and vice president of the united states. january 6, 2025 congress counts the vote congress meets in joint session to count the electoral votes (unless congress passes a law to change the date from a saturday or sunday). january 20, 2025 inauguration day the president elect is sworn in as president of the united states. view electoral college timeline this page was last reviewed on november 6, 2024. contact us with questions or comments. contact us · accessibility · privacy policy · freedom of information act · no fear act · usa.gov the u.s. national archives and records administration 1 86 nara nara or 1 866 272 6272

included are suggestions on the design, data collection, and analysis of a quality survey. for more detailed information on important details to assess rigor of survey methology, see the aapor transparency initiative. to download a pdf of these best practices, please click here “what is a survey?”, american statistical association surveys are an important research tool for learning about the feelings, thoughts, and behaviors of groups of individuals. however, surveys may not always be the best tool for answering your research questions. they may be appropriate when there is not already sufficiently timely or relevant existing data on the topic of study. researchers should consider the following questions when deciding whether to conduct a survey: surveys should not be used to produce predetermined results, campaigning, fundraising, or selling. doing so is a violation of the aapor code of professional ethics. once you have decided to conduct a survey, you will need to decide in what mode(s) to offer it. the most common modes are online, on the phone, in person, or by mail. the choice of mode will depend at least in part on the type of information in your survey frame and the quality of the contact information. each mode has unique advantages and disadvantages, and the decision should balance the data quality needs of the research alongside practical considerations such as the budget and time requirements. some surveys use multiple modes, particularly if a subset of the people in the sample are more reachable via a different mode. often, a less costly method is employed first or used concurrently with another method, for example offering a choice between online and telephone response, or mailing a paper survey with a telephone follow up with those who have not yet responded. when you run a survey, the people who respond to your survey are called your sample because they are a sample of people from the larger population you are studying, such as adults who live in the u.s. a sampling frame is a list of information that will allow you to contact potential respondents   your sample   from a population. ultimately, it’s the sampling frame that allows you to draw a sample from the larger population. for a mail based survey, it’s a list of addresses in the geographic area in which your population is located; for an online panel survey, it’s the people in the panel; for a telephone survey, it’s a list of phone numbers. thinking through how to design your sample to best match the population of study can help you run a more accurate survey that will require fewer adjustments afterwards to match the population. one approach is to use multiple sampling frames; for example, in a phone survey, you can combine a sampling frame of people with cell phones and a sampling frame of people with landlines (or both), which is now considered a best practice for phone surveys. surveys can be either probability based or nonprobability based. for decades, probability samples, often used for telephone surveys, were the gold standard for public opinion polling. in these types of samples, there is a frame that covers all or almost all the population of interest, such as a list of all the phone numbers in the u.s. or all the residential addresses, and individuals are selected using random methods to complete the survey. more recently, nonprobability samples and online surveys have gained popularity due to the rising cost of conducting probability based surveys. a survey conducted online can use probability samples, such as those recruited using residential addresses, or can use nonprobability samples, such as “opt in” online panels or participants recruited, through social media or personal networks. analyzing and reporting nonprobability based survey results often require using special statistical techniques and taking great care to ensure transparency about the methodology. if you want to measure change, don’t change the measure. to accurately measure whether an observed change between surveys taken at two points in time reflects a true shift in public attitudes or behaviors, it is critical to keep the question wording, framing, and methodology of the survey as similar as possible across the two surveys. changes in question wording and even the context of other questions before it can influence how respondents answer and make it appear that there has been a change in public opinion even if the only change is in how respondents are interpreting the question (or potentially mask an actual shift in opinion). changes in mode, such as comparing a survey conducted over the telephone to one conducted online, can sometimes also mimic a real change because many people respond to certain questions differently when speaking to an interviewer on the phone versus responding in private to a web survey. questions that are very personal or have a response option that respondents see as socially undesirable, or embarrassing are particularly sensitive to this mode effect. if changing the measure is necessary   perhaps due to flawed question wording or a desire to switch modes for logistical reasons   the researcher can employ a split ballot experiment to test whether respondents will be sensitive to the change. this would involve fielding two versions of a survey   one with the previous mode or question wording and one with the new mode or question wording   with all other factors kept as similar as possible across the two versions. if respondents answer both versions similarly, there is evidence that any change over time is likely due to a real shift in attitudes or behaviors rather than an artifact of the change in measurement. if response patterns differ according to which version respondents see, then change over time should be interpreted cautiously if the researcher moves ahead with the change in measurement. interviewers need to undergo training that covers both recruiting respondents into the survey and administering the survey. recruitment training should cover topics such as contacting sampled respondents and convincing reluctant respondents to participate. interviewers should be comfortable navigating the hardware and software used to conduct the survey and pronouncing difficult names or terms. they should have familiarity with the concepts the survey questions are asking about and know how to help respondents without influencing their answers. training should also involve practice interviews to familiarize the interviewers with the variety of situations they are likely to encounter. if the survey is being administered in languages other than english, interviewers should demonstrate language proficiency and cultural awareness. training should address how to conduct non english interviews appropriately. interviewers should be trained in protocols on how best to protect the health and well being of themselves and respondents, as needed. as an example, during the covid 19 pandemic, training in the proper use of personal protective equipment and social distancing would be appropriate for field staff. before fielding a survey, it is important to pretest the questionnaire. this typically consists of conducting cognitive interviews or using another qualitative research method to understand respondents’ thought processes, including their interpretation of the questions and how they came up with their answers. pretesting should be conducted with respondents who are similar to those who will be in the survey (e.g., students if the survey sample is college students). conducting a pilot test to ensure that all survey procedures (e.g., recruiting respondents, administering the survey, cleaning data) work as intended is recommended. if it is unclear what question wording or survey design choice is best, implementing an experiment during data collection can help systematically compare the effects of two or more alternatives. checks must be made at every step of the survey life cycle to ensure that the sample is selected properly, the questionnaire is programmed accurately, interviewers do their work properly, information from questionnaires is edited and coded accurately, and proper analyses are used. the data should be monitored while it is being collected by using techniques such as observation of interviewers, replication of some interviews (re interviews), and monitoring of response and paradata distributions. odd patterns of responses may reflect a programming error or interviewer training issue that needs to be addressed immediately. it is important to monitor responses and attempt to maximize the number of people who respond to your survey. if very few people respond to your survey, there is a risk that you may be missing some types of respondents entirely, and your survey estimates may be biased. there are a variety of ways to incentivize respondents to participate in your survey, including offering monetary or non monetary incentives, contacting them multiple times in different ways and at different times of the day, and/or using different persuasive messages. interviewers can also help convince reluctant respondents to participate. ideally, reasonable efforts should be made to convince both respondents who have not acknowledged the survey requests as well as those who refused to participate. analyzing survey data is, in many ways, similar to data analysis in other fields. however, there are a few details unique to survey data analysis to take note of. it is important to be as transparent as possible, including about any statistical techniques used to adjust the data. depending on your survey mode, you may have respondents who answer only part of your survey and then end the survey before finishing it. these are called partial responses, drop offs, or break offs. you should make sure to indicate these responses in your data and use a value to indicate there was no response. questions with no response should have a different value than answer options such as “none of the above,” “i don’t know,” or “i prefer not to answer.” the same applies if your survey allows respondents to skip questions but continue in the survey. a common way of reporting on survey data is to show cross tabulated results, or crosstabs for short. crosstabs are when you show a table with one question’s answers as the column headers and another question’s answers as the row names. the values in the crosstab can be either counts   the number of respondents who chose those specific answers to those two questions   or percentages. typically, when showing percentages, the columns total to 100%. analyzing survey data allows us to estimate findings about the population under study by using a sample of people from that population. an industry standard is to calculate and report on the margin of sampling error, often shortened to the margin of error. the margin of error is a measurement of confidence in how close the survey results are to the true value in the population. to learn more about the margin of error and the credibility interval, a similar measurement used for nonprobability surveys, please see aapor’s margin of error resources. ideally, the composition of your sample would match the population under study for all the characteristics that are relevant to the topic of your survey; characteristics such as age, sex, race/ethnicity, location, educational attainment, political party identification, etc. however, this is rarely the case in practice, which can lead to the results of your survey being skewed. weighting is a statistical technique to adjust the results to adjust the relative contributions of your respondents to match the population characteristics more closely. learn more about weighting. because there are so many different ways to run surveys, it’s important to be transparent about how a survey was run and analyzed so that people know how to interpret and draw conclusions from it. aapor’s transparency initiative has established a list of items to report with your survey results that uphold the industry transparency standards. these items include sample size, margin of sampling error, weighting attributes, the full text of the questions and answer options, the survey mode, the population under study, the way the sample was constructed, recruitment, and several other details of how the survey was run. the list of items to report can vary based on the mode of your survey   online, phone, face to face, etc. organizations that want to commit to upholding these standards can also become members of the transparency initiative. it is important to monitor responses and attempt to maximize the number of people who respond to your survey. if very few people respond to your survey, there is a risk that you may be missing some types of respondents entirely, and your survey estimates may be biased. there are a variety of ways to incentivize respondents to participate in your survey, including offering monetary or non monetary incentives, contacting them multiple times in different ways and at different times of the day, and/or using different persuasive messages. interviewers can also help convince reluctant respondents to participate. ideally, reasonable efforts should be made to convince both respondents who have not acknowledged the survey requests as well as those who refused to participate. 1436 duke street alexandria, va 22314 [email protected] | 703 827 1887 david dutwin may 2019 “many of you know me primarily as a methodologist. but in fact, my path to aapor had nothing to do with methodology. my early papers, in fact, wholly either provided criticism of, or underscored the critical value of, public opinion and public opinion polls. and so in some respects, this presidential address is for me, completes a full circle of thought and passion i have for aapor, for today i would like to discuss matters pertaining to the need to reconsider, strengthen, and advance the mission of survey research in democracy. historically, there has been much to say on the role of public opinion in democracy. george gallup summarized the role of polls quite succinctly when he said, “without polls, [elites] would be guided only by letters to congressmen, the lobbying of pressure groups, and the reports of political henchmen.” further, democratic theory notes the critical, if not the pivotal role, of public opinion and democratic practice. storied political scientist v.o key said: “the poll furnishes a means for the deflation of the extreme claims of pressure groups and for the testing of their extravagant claims of public sentiment in support of their demands.” furthermore, surveys provide a critical check and balance to other claims of what the american public demands in terms of policies and their government. without polls, it would be all that much harder to verify and combat claims of public sentiment made by politicians, elites, lobbyists, and interest groups. [“no policy that does not rest upon some public opinion can be permanently maintained.”  abe lincoln; “public opinion is a thermometer a monarch should constantly consult”   napoleon] it is sometimes asked whether leaders do consult polls and whether polls have any impact of policy. the relationship here is complex, but time and again researchers have found a meaningful and significant effect of public opinion, typically as measured by polling, on public policy. as one example, page and shapiro explored trends in american public opinion from the 1930s to the 1980s and found no less than 231 different changes in public policy following shifts in public opinion. and certainly, in modern times around the world, there is recognition that the loss of public opinion would be, indeed, the loss of democracy itself. [“where there is no public opinion, there is likely to be bad government, which sooner or later, becomes autocratic government.”   willian lyon mackenzie king] and yet, not all agree. some twist polling to be a tool that works against democratic principles. [“the polls are just being used as another tool for voter suppression.”   rush limbaugh] and certainly, public opinion itself is imperfect, filled with non attitudes, the will of the crowd, and can often lead to tyranny of the majority, as jon stewart nicely pointed out. [“you have to remember one thing about the will of the people: it wasn’t that long ago that we were swept away by the macarena.”   jon stewart] if these later quotes were the extent of criticism on the role of public opinion and survey research in liberal democracy, i would not be up here today discussing what soon follows in this address. unfortunately, however, we live a world in which many of the institutions of democracy and society are under attack. it is important to start by recognizing that aapor is a scientific organization. whether you are a quantitative or qualitative researcher, a political pollster or developer of official statistics, a sociologist or a political scientist, someone who works for a commercial entity or nonprofit, we are all survey scientists, and we come together as a great community of scientists within aapor, no matter our differences. and so we, aapor, should be as concerned as any other scientific community regarding the current environment where science is under attack, devalued, and delegitimized. it is estimated that since the 2016 election, the federal policy has moved to censor, or misrepresent, or curtail and suppress scientific data and discoveries over 200 times, according to the sabin center at columbia university. not only is this a concern to aapor as a community of scientists, but we should be concerned as well on the impact of these attacks on public opinion itself. just as concerning is the attack on democratic information, in general. farrell and schneier argue that there are two key types of knowledge in democracy, common and contested. and while we should be free to argue and disagree with policy choices, our pick of democratic leaders, and even many of the rules and mores that guide us as a society, what is called contested knowledge, what cannot be up for debate is the common knowledge of democracy, for example, the legitimacy of the electoral process itself, or the validity of data attained by the census, or even more so, i would argue, that public opinion does not tell us what the public thinks. as the many quotes i provided earlier attest to, democracy is dependent upon a reliable and nonideological measure of the will of the people. for more than a half century and beyond, survey research been the principal and predominant vehicle by which such knowledge is generated. and yet, we are on that doorstep where common knowledge is becoming contested. we are entering, i fear, a new phase of poll delegitimization. i am not here to advocate any political ideology and it is critical for pollsters to remain within the confines of science. yet there has been a sea change in how polls are discussed by the current administration. to constantly call out polls for being fake is to delegitimize public opinion itself and is a threat to our profession. worse still, many call out polls as mere propaganda (see joondeph, 2018). such statements are more so a direct attack on our science, our field, and frankly, the entire aapor community. and yet even worse is for anyone to actually rig poll results. perhaps nothing may undermine the science and legitimacy of polling more. more pernicious still, we are on the precipice of an age where faking anything is possible. the technology now exists to fake actual videos of politicians, or anyone for that matter, and to create realistic false statements. the faking of poll results is merely in lockstep with these developments. there are, perhaps, many of you in this room who don’t directly connect with this. you do not do political polling. you do government statistics. sociology. research on health, on education, or consumer research. but we must all realize that polling is the tip of the spear. it is what the ordinary citizen sees of our trade and our science. as andy kohut once noted, it represents all of survey research. [political polling is the “most visible expression of the validity of the survey research method.“   andrew kohut] with attacks on science at an all time high in the modern age, including attacks on the science of surveys; with denigration of common knowledge, the glue that holds democracy together, including denunciation on the reliability of official statistics; with slander on polling that goes beyond deliberation on the validity of good methods but rather attacks good methods as junk, as propaganda, and as fake news; and worse of all, a future that, by all indications, will if anything include the increased frequency of fake polls, and fake data, well, what are we, aapor, to do? we must respond. we must react. and, we must speak out. what does this mean, exactly? first, aapor must be able to respond. specifically, aapor must have vehicles and avenues of communication and the tools by which it can communicate. second, aapor must know how to respond. that is to say, aapor must have effective and timely means of responding. we are in an every minute of the day news cycle. aapor must adapt to this environment and maximize its impact by speaking effectively within this communication environment. and third, aapor must, quite simply, have the willpower to respond. aapor is a fabulous member organization, providing great service to its members in terms of education, a code of ethics, guidelines for best practices and promotions of transparency and diversity in the field of survey research. but we have to do more. we have to learn to professionalize our communication and advocate for our members and our field. there are no such thing as sidelines anymore. we must do our part to defend survey science, polling, and the very role of public opinion in a functioning democracy. this might seem to many of you like a fresh idea, and bold new step for aapor. but in fact, there has been a common and consistent call for improved communication abilities, communicative outreach, and advocacy by many past presidents, from diane colasanto to nancy belden to andy kohut. past president frank newport for example was and is a strong supporter of the role of public opinion in democracy, underscoring in his presidential address that quote, “the collective views of the people…are absolutely vital to the decision making that ultimately affects them.” he argued in his presidential address that aapor must protect the role of public opinion in society. a number of past presidents have rightly noted that aapor must recognize the central role of journalists in this regard, who have the power to frame polling as a positive or negative influence on society. president nancy mathiowetz rightly pointed out that aapor must play a role in, and even financially support, endeavors to guarantee that journalists’ support aapor’s position on the role of polling in society and journalists’ treatment of polls. and nancy’s vision, in fact, launched relationship with poynter in building a number of resources for journalist education of polling. past president scott keeter also noted the need for aapor to do everything it can to promote public opinion research. he said that “we all do everything we can to defend high quality survey research, its producers, and those who distribute it.” but at the same time, scott noted clearly that, unfortunately, “at aapor we are fighting a mostly defensive war.” and finally, past president cliff zukin got straight to the point in his presidential address, noting that, quote “aapor needs to increase its organizational capacity to respond and communicate, both internally and externally. we need to communicate our positions and values to the outside world, and we need to diffuse ideas more quickly within our profession.” aapor is a wonderful organization, and in my biased opinion, the best professional organization i know. how have we responded to the call of past presidents? i would say, we responded with vigor, with energy, and with passion. but we are but a volunteer organization of social scientists. and so, we make task forces. we write reports. these reports are well researched, well written, and at the same time, i would argue, do not work effectively to create impact in the modern communication environment. we have taken one small step to ameliorate this, with the report on polling in the 2016 election, which was publicly released via a quite successful live facebook video event. but we can still do better. we need to be more timely for one, as that event occurred 177 days after the election, when far fewer people were listening, and the narrative was largely already written. and we need to find ways to make such events have greater reach and impact. and of course, we need more than just one event every four years. i have been proud to have been a part of, and even be the chair of, a number of excellent task force reports. but we cannot, i submit, continue to respond only with task force reports. aapor is comprised of the greatest survey researchers in the world. but it is not comprised of professional communication strategists, plain and simple. we need help, and we need professional help. in the growth of many organizations, there comes a time when the next step must be taken. the asa many years ago, for example, hired a full time strategic communications firm. other organizations, including the nca, apsa, and others, chose instead to hire their own full time professional communication strategist. aapor has desired to better advocate for itself for decades. we recognize that we have to get into the fight, that there are again no more things as sidelines. and we have put forward a commendable effort in this regard, building educational resources for journalists, and writing excellent reports on elections, best practices, sugging and frugging, data falsification, and other issues. but we need to do more, and in the context of the world outside of us, we need to speak a language that resonates with journalists, political elites, and perhaps most importantly the public. i want to stop right here and make it clear, that the return on investment on such efforts is not going to be quick. and the goal here is not to improve response rates, though i would like that very much! no, it is not likely that any efforts in any near term reverses trends in nonresponse. it may very well be that our efforts only slow or at best stop the decline. but that would be an important development. the washington post says that democracy dies in darkness. if i may, i would argue that aapor must say, democracy dies in silence, when the vehicle for public opinion, surveys, has been twisted to be distrusted by the very people who need it most, ordinary citizens. for the most part, aapor has been silent. we can be silent no more. this year, executive council has deliberated the issues outlined in this address, and we have chosen to act. the road will be long, and at this time, i cannot tell you where it will lead. but i can tell you our intentions and aspirations. we have begun to execute a 5 point plan that i present here to you. first, aapor executive council developed and released a request for proposals for professional strategic communication services. five highly regarded firms responded. after careful deliberation and in person meetings with the best of these firms, we have chosen stanton communications to help aapor become a more professionalized association. our goals in the short term are as follows. we desire to become more nimble and effective at responding to attacks on polls, with key aapor members serving as spokespersons when needed, but only after professional development of the messages they will promulgate, approved by council, and professionalized by the firm. stanton brings with it a considerable distribution network of journalists and media outlets. aapor, through its professional management firm kellen, has access to audio and video services of the national press club, and will utilize these services when needed to respond to attacks on polls, and for other communications deemed important by aapor executive council. our plan is to begin small. we are cognizant of the cost that professional communication can entail, and for now, we have set very modest goals. the first step is to be prepared, and have a plan for, the 2020 election, with fast response options of communication during the campaign, and perhaps most importantly, directly thereafter. the second element of our plan is to re envision aapor’s role in journalism education. in short, we believe we need to own this space, not farm it out to any other entity. we need refreshed educational videos, and many more of them, from explaining response rates to the common criticisms made on the use of horserace polling in the media. we need to travel. willing aapor members should be funded to travel and present at journalism conferences, to news rooms, and to journalism schools on an annual basis. aapor could as well have other live events, for example a forum on the use of polls in journalism. there should be a consistent applied effort over time. the media and journalists are aapor’s greatest spokespeople. by and large, much of our image is shaped through them. the third element looks at the long game. and that is, for aapor to help in developing civics education on public opinion and the role of public opinion in democracy. with the help of educational experts, and importantly, tipping our hats to our aapor’s got talent winner last year, allyson holbrook, who proposed exactly this kind of strategy, we believe aapor can help develop a curriculum and educational materials and engage with educators to push for the inclusion of this curriculum in primary education. aapor can and should develop specific instructional objectives of civics education by grade and develop a communications plan to lobby for the inclusion of this civics curriculum by educators. the fourth element is for aapor to direct the transparency initiative to develop a strategic plan for the next ten years. we recognize that it is not always the case that polls are executed with best practices. how does aapor respond in these instances? with a plethora of new sampling approaches and modalities, we believe the ti needs to have a full throated conversation about these challenges and how aapor should handle them. after all, this too is part of the conversation of aapor communication. finally, aapor should, as past president tim johnson called for last year, learn as much as it can about the perceptions of polls in society. we cannot make effective strategic communication plans without knowing first how they will resonate and know to some degree their expected effectiveness. such an effort should continue over time, building both a breadth and depth of understanding. if this sounds a bit like a wish list, well, you would be right. for now, the immediate goal for aapor and its communication firm is to prepare for 2020 and to take some modest steps toward professionalizing aapor’s ability to effectively and quickly communicate and advocate. looking toward the future, aapor council has authorized the development of the ad hoc committee on public opinion. this committee will be comprised of aapor members dedicated to pushing forward this agenda. we recognize the potential cost of these endeavors in terms of money and labor, and so in each area, there will be mission leaders on the committee whose goal is to push forward with two goals. the first is funding. we cannot and should not fund these endeavors alone. we will be seeking foundational funding for each of these areas, and are developing a proposal for each specifically. perhaps only one area attains funding, perhaps all of them. no matter, the committee will adjust its goals contingent on the means it has available. a number of members have already asked to be part of these efforts. but i call on all of you, the aapor membership, to reach out and join the effort as well. we need people experienced in seeking funding, and people passionate in moving the needle with regard to polling journalism, civics education, and the role of public opinion in democracy. aapor’s secret sauce has always been the passion of its members and we call on you to help. please go the link below to tell us you want to join the effort. friends and colleagues, one of the many excellent aapor task forces already, in fact explored this issue, the task force on polling and democracy and leadership. they argued that “aapor should adopt an increased public presence arguing for the importance of public opinion in a democracy and the importance of rigorous, unbiased, scientific research assessing public opinion.” it is time we strive to realize these aspirations. for the good of our association, our field, and our very democracy. if past efforts by aapor volunteers are any indication, we anticipate great success and health in the future of our field and our endeavors. it has been an honor and a privilege serving as your president. thank you.”

when designing surveys, creators often spend hours fine tuning each question to ensure clarity, avoid bias, and guarantee questions align with research objectives. however, one frequently overlooked element that can significantly impact success is the order of survey questions. for survey creators, the arrangement of questions isn’t just an afterthought it’s a critical factor in ensuring the accuracy and validity of the data collected. this blog will delve into how survey question order can affect responses, the different psychological and cognitive biases that come into play, and how to structure questionnaires for maximum effectiveness. how people respond to surveys is influenced by a variety of factors, one of the most important being question order. psychological research has shown that the order of questions in a questionnaire can introduce certain biases that affect how respondents interpret subsequent questions. imagine a survey about job satisfaction. if the first question asks, “how satisfied are you with your work life balance?” and the second asks, “how satisfied are you with your job overall?” respondents might unconsciously link these two questions. they might assume that job satisfaction is inherently tied to work life balance and adjust their answers accordingly. alternatively, the responses might reflect a different pattern if the survey started with a broader question about overall job satisfaction and then drilled down into specific aspects like work life balance. for survey creators, understanding these subtle biases is crucial for designing effective surveys. if you want to know more about biased surveys, read our blog post about them. when designing a survey, understanding the different types of order effects like primacy, recency, carryover, and contrast is essential. these subtle biases can significantly shape how respondents answer questions and impact the overall accuracy of data. this refers to respondents’ tendency to favor information presented first. in surveys, this can manifest when respondents give more weight to initial questions, which affects their responses to later ones. if a survey begins with a complex or emotionally charged question, it can set the tone for the entire survey. respondents may also be more inclined to think critically about early questions while providing less thoughtful responses to those that appear later. on the flip side, the recency effect is when respondents give undue importance to the last question or option encountered. this is particularly relevant when respondents are asked to evaluate a list of items. for example, in a rank order survey, the items that appear at the bottom of the list might be remembered better, skewing the ranking. this occurs when earlier questions influence how respondents interpret later ones. for example, if the first survey question asks about a positive experience with a product, and the following questions are about specific product features, respondents might rate those features more positively than they would otherwise have. survey creators need to be aware of how the sequence of questions can create unintended contexts that affect the responses to later questions. this comes into play when a respondent’s answer to one question is influenced by the stark difference between it and the previous one. for example, if a survey asks about an impactful life event and then follows up with a question about a minor inconvenience, respondents may downplay the latter’s importance simply because it pales in comparison. the challenge for survey creators is to recognize these order effects and, depending on the research goals, structure surveys to avoid or mitigate them. as a survey creator, you’re not just asking questions but crafting a journey for respondents. this can profoundly impact the quality and accuracy of the data collected. below are several strategies for ordering questions to maximize engagement and minimize bias. starting surveys with straightforward, easy to answer questions can help put respondents at ease. demographic questions, such as age, gender, or occupation, are good warm ups, being seen as neutral and non invasive, which helps ease respondents into the survey. by contrast, opening with personal or complex questions can create discomfort, leading to higher abandonment rates. clustering related questions together can create a more cohesive and logical flow for the respondent. for instance, if conducting a customer satisfaction survey, group questions about product features, then follow these with questions about customer service and overall satisfaction. this method helps respondents focus on one aspect of their experience at a time, reducing the risk of confusion or cognitive overload. the funnel approach is a widely used method that begins with broad, general questions that gradually move toward more specific, detailed ones. for instance, in a market research survey, start by asking respondents about how familiar they are with your brand, then follow with more specific questions about particular products or services. the funnel method lets you gather general insights before diving into more detailed data. alternatively, some survey creators prefer the pyramid approach, where specific questions are the beginning point, then a broadening to more general ones follows. this method can work well in situations where you want respondents to reflect on particular details before providing an overall judgment. for example, asking about specific features and then moving on to general satisfaction in a product feedback survey can lead to more thoughtful, accurate responses. questions that delve into personal or sensitive topics such as income, political views, or health status should be placed toward the end of the survey. by the time respondents reach these questions, they will have invested time and effort into the survey, making them less likely to abandon it. moreover, rapport has been built, making them more comfortable sharing sensitive information. randomizing the order of survey questions or options can help mitigate order effects like primacy and recency for certain types of surveys, particularly those involving ranking or rating multiple items. for example, if you ask respondents to rank a list of favorite products, randomizing the list for each respondent can ensure that no single one is favored due to its position. using brief transition statements between sections can help maintain respondent engagement and clarity if a survey covers multiple topics. for instance, if a survey moves from questions about personal behavior to questions about opinions, a transition like “next, we’d like to ask you some questions about your opinions on recent industry trends” helps orient the respondent. survey creators should always appreciate the importance of pilot testing. trying out different question ordering on a small subset of the target audience allows any issues with clarity, engagement, or response patterns to be spotted. you may discover, for instance, that respondents are more likely to abandon the survey if sensitive questions are placed too early or that rank order survey questions lead to biased results if the options aren’t randomized. moreover, testing can reveal whether respondents are experiencing cognitive fatigue or confusion as they progress through the survey. by analyzing such feedback, questionnaires can be refined and necessary adjustments made to the structure and order of questions before forwarding it to the entire sample audience. while following best practices for structuring surveys is important, flexibility is also crucial. no two surveys are alike and different objectives may require different approaches to question ordering. for instance, if conducting a survey requires highly detailed responses, you may want to place those questions earlier when respondents are more engaged. on the other hand, if the survey is long, including some simpler questions throughout can help break up the mental load. with the rise of online survey tools, survey creators have more options than ever for experimenting with question order. most modern survey platforms allow for quickly shuffling question order, implementing logic flows (where the response to one question determines the next), and even personalizing question paths for different respondents. by leveraging these technological tools, a more dynamic and engaging survey experience can be created while the risk of order effects is minimized. for survey creators, the order of questions in a questionnaire is not just a logistical consideration it’s a critical aspect of survey design that directly impacts data quality. whether conducting market research, employee engagement surveys, or customer satisfaction assessments, paying attention to the order of survey questions can help avoid common pitfalls like primacy and recency effects while ensuring that data is accurate, reliable, and actionable. ultimately, taking the time to thoughtfully arrange questions, pilot test, and refine a survey will pay off in more meaningful insights and better decision making. so, the next time you sit down to create a survey, remember that it’s not just about the questions asked it’s also about how and when they are asked. by understanding and mastering the art of question order, you can elevate surveys and the insights they provide, making you a more effective and informed survey creator. ready to create your next survey with precision and ease? with surveyplanet, you can craft surveys that engage respondents while ensuring high quality, unbiased data. our intuitive platform offers features like customizable question order, randomization, and seamless logic flows to help get the most accurate insights. whether designing a simple questionnaire or a complex survey, surveyplanet makes it easy to put best practices into action. start creating surveys for free with surveyplanet today and elevate your data collection process! photo by martin adams on unsplash © 2025 survey planet, llc. all rights reserved

i’ve spent more than a decade at the times, covering stories for nearly every desk, including breaking news, the presidency of donald j. trump, gun violence in america and the last four olympic games. i’ve reported from six countries and more than a dozen states. i’ve skied with mikaela shiffrin (i fell), been on a balance beam with simone biles (i almost fell), covered a hurricane from an ihop in orlando and flown a drone over the panama canal. my work often involves creating data sets where they didn’t exist before, like watching and categorizing all 1,150 episodes of “tucker carlson tonight”; reading each of the 11,390 tweets trump posted while in office; and aggregating more than 40,000 reader submissions to create an extremely detailed map of new york city’s neighborhoods. i’ve always been interested in the intersection of art and journalism and how we can use both to better explain the world. at indiana university, i merged a journalism major with a fine arts minor, taking classes in news writing, sculpture and color theory. before working for the times, i reported and made interactive graphics for the new yorker. i have also taught journalism at universities for nearly a decade, most recently in the graduate program at columbia university. i have won two emmy awards for my work on shows about two extremely different television hosts: bob ross and tucker carlson. in addition to adhering to the standards set out in the times ethical journalism handbook, visuals have their own set of ethical standards. how should we design an arrow that represents troop movements that visually matches what our reporting shows? which republican candidate in a crowded primary field gets to be represented by the color red on a map? does this data want to be a line chart or a bar chart   or something else? these may, at first, all sound like aesthetic questions, but at their core they are journalistic ones. and they all have a different set of ethical questions we discuss on a case by case basis. when producing visualizations, i always try to consult with multiple sources about the limits and caveats of the data. i ask for the raw data and try to recreate visualizations whenever possible. i annotate and explain what we want a reader to notice. i make sure the text and the visuals work together and also give context, much like a painting hanging in a museum and the text on the wall that accompanies it. i try to actually show the thing i am describing or writing about, whenever i can. clarity is the guiding principle of my work; aesthetics are secondary. hearing from readers is always a treat (even if you don’t like something). i try to respond to all the mail i get. email: larry.buchanan@nytimes.com x: @larrybuch instagram: @larrybuch we’d like you to look at one piece of art for 10 minutes, uninterrupted. by larry buchanan the trump administration ordered temporary freezes in funding for programs spanning virtually every part of the government. here’s the full list. by the upshot staff we’d like you to look at one piece of art for 10 minutes, uninterrupted. by larry buchanan local election workers share how they would respond to a range of scenarios, using a 52 card game. by alicia parlapiano, francesca paris, ben blatt and larry buchanan donald trump has softened his rhetoric, contradicted himself and nearly dropped “pro life” from his vocabulary. yet there is no evidence that his views on abortion have changed. by margot sanger katz, claire cain miller and eve washington new york and cities across the country reconsider decades old parking rules. by emily badger, mihir zaveri and larry buchanan we’d like you to look at one piece of art for 10 minutes, uninterrupted. by larry buchanan, francesca paris and nico chilla we’d like you to look at one piece of art for 10 minutes, uninterrupted. by larry buchanan, nico chilla and francesca paris we’d like you to look at one piece of art for 10 minutes, uninterrupted. by larry buchanan, nico chilla and francesca paris we’d like you to look at one piece of art for 10 minutes, uninterrupted. by larry buchanan, francesca paris and nico chilla

official website of the commonwealth of pennsylvania local, state, and federal government websites often end in .gov. commonwealth of pennsylvania government websites and email systems use "pennsylvania.gov" or "pa.gov" at the end of the address. before sharing sensitive or personal information, make sure you're on an official state website. there have been a lot of changes with pennsylvania elections. getting clear, accurate information from a trusted source is critical to ensuring that every eligible voter can cast their vote and make their voice heard. all registered pa voters can cast their ballot by mail. apply online today. applications are available in multiple languages. you do not need an excuse to apply for a mail in ballot. vote at your county election office. once your county’s ballots are finalized and printed, you can go to your county election office, apply for a mail ballot, vote your ballot, and submit your ballot in the same visit. vote in person at your polling place on election day. polls will be open from 7 a.m. to 8 p.m. on election day. find your polling place at vote.pa.gov/ no. unless it's your first time voting at a new polling place. if it's your first time voting at a new location, you must bring one of the approved forms of identification. your id doesn't need to be a photo id. there are many forms of non photo id that are acceptable. call the department of state's election hotline at 1 877 votes pa for assistance. telephone interpreters are available. it depends on whether you asked to be an annual voter on your primary ballot application. check the status of your mail ballot. yes. if a voter is unable to complete their mail in or absentee ballot independently, they may designate someone in writing using the authorize a designated agent form or a form provided by your county to help them. pennsylvania has also implemented an accessible remote ballot marking solution for mail voting for use by voters with a disability. if you need assistance, you may call 1 877 votes pa (1 877 868 3772). yes. if you don't want to receive mail ballots for a specific year, just don't return the annual application you receive from your county. you'll be able to vote in person, on election day, using a regular ballot. yes. you will receive an application every year until you cancel your annual mail in or absentee status or move to another county and do not request to transfer your status to the new county. yes. if you move, you should update your address by submitting a voter registration form. you can check “transfer” on the voter registration form to indicate you would like to maintain your annual mail ballot status. contact your county board of elections if you have additional questions or need assistance. you must go to your correct polling place to vote. however, if you believe you are eligible to vote at that polling place, you may vote using a provisional ballot. yes. if this happens, you have the right to vote using a provisional ballot. no. the department of state has confidence in the security of pennsylvania's election process and mail ballot system. additional details: each mail ballot packet sent to a voter has a unique id number. that same number is also printed on the pre addressed outer return envelope where the voter must sign. when either type of mail ballot (absentee or no excuse mail in) is received and pre canvassed, the unique id number is scanned into the statewide system. to issue a second ballot to a voter, the county must first cancel the ballot initially issued to the voter. in other words, each voter can receive only one ballot. if a voter attempts to vote a second ballot, the system will flag that there is already a ballot from that voter. if a voter returns a mail ballot, their record is marked, and that notation is included in the poll book to ensure that poll workers are aware that the voter's ballot has been cast. the department also updated the poll book format to move the names of voters who have returned their ballots to a separate list at the end of the poll book. those voters can only vote at a polling place by means of a provisional ballot, which is reviewed by the county after the election. if the county determines that a voter did, in fact, already cast a mail ballot, the voter's provisional ballot will not be counted. counting the hundreds of thousands   sometimes millions   of mail ballots cast in every election takes time. unlike other states, state law in pennsylvania prevents counties from even beginning to open ballots until 7 a.m. on election day   when those same election officials are managing in person voting at more than 9,100 polling places. that law can only be changed if the pennsylvania legislature sends a bill to the governor’s desk to allow counties to open mail ballot envelopes, flatten the ballots, and prepare them to be tallied before election day as most other states already allow. while we know there may be an eagerness to predict the outcomes, counties are prioritizing accuracy and security as they count ballots. the results of federal and state elections in pennsylvania are available on our election results website starting after 8 p.m. on election night. the results of county, city, and local elections in pennsylvania may be found on your county's board of elections website. finally, when it comes to results and any other election related information, make sure you are getting your information from trusted sources, such as the department of state's voting website, vote.pa.gov. find more services and information about voting in pennsylvania.

an official state of kansas government website. here's how you know. the .gov means it's official.a .gov website belongs to an official government organization in the united states. the site is secure.the https:// or lock icon ensures you're safely connected to the website and any information you provide is encrypted. q. do i have to live in kansas for a certain amount of time to be eligible to vote? a. no, kansas does not have a length of residence requirement. however, you have to be registered 21 days prior to an election and be a resident of kansas at the time of registration. q. can i pre register if i will be 18 years old before the next election? a. if you are 17 years old and will be 18 before the next election, you may apply to register to vote. individuals are not allowed to vote in any election if they are not registered to vote and 18 years of age. q. what are the disqualifications for voting in kansas? a. individuals may be found disqualified to vote by reason of death, federal or state felony conviction or by declaration of a kansas court of law. q. i have a felony conviction, can i vote? a. a federal or state felony conviction results in the loss of voting rights until the terms of the felony sentence is complete. if on probation or parole, a sentence is not considered complete until your probation or parole is finished. individuals who lose their voting rights due to a felony conviction may re register to vote upon completion of their sentence. the voter registration form includes an affidavit above the signature line attesting an individual’s right to vote has been restored. signing a false affidavit is a felony and could result in the loss of voting rights, if convicted. q. what if my certificate of registration has incorrect information? a. after registering to vote, you should receive a certificate of registration from your local county election officer. if any information on the certificate is incorrect, please contact your local election office immediately. q. i moved to a new address in the county i’m currently registered in, can i vote? a. individuals who move to a new address within the same county should update their information as soon as possible. if you move after the registration deadline, you may vote at the precinct assigned to your old address, but you will be required to complete a new voter registration application. q. i moved from one county to another county in kansas, how do i vote? a. if you move to a new county in kansas before the registration deadline, you must complete a new voter registration application. if you move after the registration deadline, you can still vote at the precinct assigned to your old address, but you will be required to complete a new voter registration application. q. i moved to kansas from another state, when can i vote? a. you should register to vote in kansas as soon as possible after moving. if 45 days before an election and the deadline to register has passed, you may vote a president only ballot. to vote a president only ballot, you must go to your local election office by noon on the day before the election and complete voter registration forms. q. can i review a sample ballot prior to voting? a. yes! you can view sample ballots online starting 45 days prior to election or you may request a sample ballot at your local election office or at your polling place. to view sample ballots online, visit: myvoteinfo.voteks.org q. what if i’m still in line when polls close? a. stay in line! all voters who are in line when polling locations close are allowed to vote. q. what should i expect when voting in kansas? a. to vote at a polling place, either on or before election day, you can expect to do the following: q. what if i make a mistake, can i request another ballot? a. yes, voters may obtain up to two replacement ballots if there is mismarked or spoiled ballots. q. i live overseas and applied to cast a federal services voter ballot. how long is my application good for? a. ballot applications from qualified federal service voters are good for the calendar year in which it was submitted. q. how do i get my ballot if i live overseas and cast a ballot under the federal voting assistance program (uovaca)? a. voters who participate in the federal voting assistance program choose their preferred method of ballot delivery   mail, fax or email   when they complete their application. we recommend individuals who choose to cast a ballot via email return it to their local election office through an encrypted email. q. do all kansas counties have to offer advance in person voting? a. yes. state law allows in person advance voting to begin up to 20 days before election day at county election offices or satellite voting locations. all counties must offer in person advance voting no later than one week before election day. q. do i have to show photo id to advance in person vote? a. state law requires kansas voters to show photographic identification when casting a vote in person, this includes voting advance in person. if the voter does not provide identification or it is not valid, the voter will be issued a provisional ballot. q. what is the difference between all mail ballot elections and advance by mail ballots? a. kansas only conducts an election entirely by mail, without the use of polling places, in question submitted elections only. k.s.a. 25 432(d) prohibits all mail elections when a candidate appears on the ballot. advance voting by mail has been allowed to all kansas voters since 1996. you do not need an excuse to advance vote by mail ballot. however, you do need to apply to have an advanced by mail ballot mailed to you for each election as they are not mandatory and are sent to voters by county election offices at the request of the voter. q. is it safe to cast an advance by mail ballot? a. kansas has several key security provisions in place for verifying the legitimacy of advance by mail ballots and those who cast them. they include: q. how can i find out the status of my advance by mail ballot application and/or ballot? a. voters can call their local election office to check on the status of their advance by mail ballot application and/or ballot. in addition, for the first time, voters can track the status of their advance by mail ballot application and advance by mail ballot online at myvoteinfo.voteks.org. q. i requested a mail in ballot for the election. can i vote in person or do i need to fill out my mail in ballot? a. voters who requested an advance by mail ballot but choose to cast their ballot in person will be given a provisional ballot to ensure they only vote once. q. what types of photo id are acceptable in kansas? do exemptions exist? a. state law requires voters to show photographic identification when casting a vote in person. if the photo id has an expiration date on it, the id must not have expired at the time of voting. an acceptable photo id does not have to have an expiration date on the document in order to be valid. acceptable forms of photo id include: exemptions exist for individuals 65 years of age or older who may use expired photo ids or individuals whose religious beliefs prohibit photographic identification. voters with religious objections may sign an affidavit before the election or at the poll on election day to be exempt. state photo id requirements do not apply to military and overseas citizens who vote under the federal uniformed and overseas citizens absentee voting act (uocava); for mail ballot elections; or voters who qualify, apply and are accepted to the permanent advance voting list, as long as they remain on the permanent advance voting list. q. do i have to provide photo id for mail ballot elections? a. no. photo id requirements do not apply in mail ballot elections. mail ballot elections are limited to local jurisdictions holding special question submitted elections. q. i do not have an approved government issued photo id and cannot afford to get one. what are my options? a. registered voters who do not have an approved government issued photo id and would like one to vote may apply for a free nondriver identification card with the kansas division of motor vehicles. to get a free photo id, individual must fill out a certification requesting fee waiver for nondriver identification card (form de vid1 (español)). this form is available at all driver’s licenses offices, all county election offices and online. q. i tried to get a free nondriver identification card so i can vote but i first need to get a birth certificate to prove my identity and residence. help! a. qualifying individuals who lack proof of identity and want to obtain a free nondriver identification card may get a kansas birth certificate from the kansas office of vital statistics at no cost. to qualify, individuals must: to get a free birth certificate to obtain a nondriver identification form, individual must submit the following forms to the kansas office of vital statistics: q. i was not born in kansas and cannot locate an acceptable form that shows proof of my identity but i want to vote in the upcoming election. money is tight and i cannot afford to pay for a birth certificate in another state. what are my options? a. individuals born outside of kansas, who do not have proof of identity and who do not wish to pay to obtain such a document from their home state may apply for a state voter identification document. this document can only be used to vote in kansas. to qualify for a state voter identification document, individuals must: memorial hall, 1st floor120 sw 10th avenuetopeka, ks 66612 1594 785 296 4564 kssos@ks.gov 8:00 a.m. to 5:00 p.m. 785 296 4564 kssos@ks.gov 800 262 vote(8683) election@ks.gov 785 296 book(2665) sos_lawbooks@ks.gov © 2024 kansas secretary of state

root out friction in every digital experience, super charge conversion rates, and optimize digital self service uncover insights from any interaction, deliver ai powered agent coaching, and reduce cost to serve increase revenue and loyalty with real time insights and recommendations delivered to teams on the ground know how your people feel and empower managers to improve employee engagement, productivity, and retention take action in the moments that matter most along the employee journey and drive bottom line growth whatever they’re saying, wherever they’re saying it, know exactly what’s going on with your people get faster, richer insights with qual and quant tools that make powerful market research available to everyone run concept tests, pricing studies, prototyping + more with fast, powerful studies designed by ux research experts track your brand performance 24/7 and act quickly to respond to opportunities and challenges in your market explore the platform powering experience management popular use cases customer learn company the annual gathering of the experience leaders at the world’s iconic brands building breakthrough business results, live in salt lake city. 9 min read you write a survey like you write a novel. what happens in the beginning sets the scene for how the story unfolds. here’s how question order and survey style can help you get the best quality data. in a word   absolutely. besides the building and creation, the order of your questions can have a real effect on how respondents answer. studies over the last several decades have demonstrated that question order affects responses in surveys about everything from presidential campaigns to employee opinions. data skewing created by the order of survey questions is a form of response bias   bias that results from the way a survey is designed. get started with our free survey maker tool one of the major pitfalls in survey question ordering is the risk of accidentally anchoring or priming your respondent. anchoring is where an early piece of information ‘sets the tone’ and limits or influences all of your participant’s subsequent answers. an example might be adding a dollar sign or other currency symbol to the answer box for a question on the value of their spouse. this will steer the participant towards answering not only this question, but other questions, in terms of monetary value rather than personal qualities. in a questionnaire, anchoring could accidentally be established by the nature and wording of prior questions or by the survey introduction, creating bias later on in the questionnaire. priming is even easier to do by accident. it happens when someone is influenced by an idea, maybe an unrelated one, that has been introduced before they answer the question. for example, say you set a question that asks respondents for their three favorite vacation destinations, just after you ask them a question about their liking for french food. will france be more likely to make the dream vacation list? priming theory suggests so. priming can be very hard to avoid in survey questionnaires, since questions must be answered sequentially and early items are almost bound to have an effect on the subject’s state of mind when they come to answer later questions. later, we’ll introduce a technique for minimizing priming effects. primacy and recency bias theory (aka the serial position effect) says that the first and last items will receive the most attention from an individual. the first item (primacy) is salient because it falls at the beginning of an experience when the person is most alert and is learning and understanding the nature of the task. the final item (recency) lingers in the memory because it’s the last thing the person sees when they complete the survey. so according to this theory, the first and last questions in your survey will have the most impact and receive the most attention, while the material in the middle of your questionnaire is more vulnerable to being rushed or skipped over. primacy and recency can also have an effect on answer choice in e.g. multiple choice questions. people love to be consistent. assimilation contrast theory holds that a person’s judgement of something can act as a type of anchor, influencing their later judgements. once someone makes a judgement, they’ll become biased towards maintaining their point of view, “assimilating” other neutral information in a biased way so that it seems to support their ideas. if something they see or read goes against their anchored judgement, they will be biased against it   this is the “contrast” part of the equation. what this means for the ordering of survey questions is that if you ask someone to take up a position, agree or disagree with something early in the questionnaire, their later answers will be colored by this judgement, and their responses may be polarized in favor of ideas that support the judgement, and in rejection of things that don’t. a similar effect is described as additive and subtractive question order bias (moore, gallup 2002). much as we love them, surveys can be taxing to take, especially when a respondent is tired, distracted, or otherwise disengaged. you can typically expect to see a person’s attention wane during a task like survey taking after a peak of attention at the beginning. in some cases, people may become bored or distracted before the end of the survey and drop out of it altogether, failing to finish. this has an obvious relevance to question order, since what comes last may be most at risk of being overlooked or ignored. although we want to maximize the attention window at the start of survey taking, we don’t want to be too familiar too quickly. the start of a survey may also be the start of a relationship with a new respondent, so it’s a good idea to begin with easy, enjoyable and non controversial questions. start with broad general interest questions that are easy for the respondent to answer. these questions serve to warm up the respondent and get them involved in the survey. by starting off with questions that promote a sense of ease and positivity, you’re laying the foundation for the respondent being willing to answer more complex and even sensitive questions later on. we know that priming effects are almost always present in surveys, but fortunately you can offset the effect to some extent by randomizing your questions so that different groups of participants get them in different orders. you may not want to randomize the whole questionnaire, as this could be confusing for participants, but you can randomize groups of related questions so that there’s still some logical order. some experts suggest that it’s best to leave any questions that may cause offence or seem intrusive until the end of your survey, so that they don’t bias the earlier responses or cause the participant to want to abandon the survey. this makes sense from the perspective of consistency and assimilation too, since someone who has observed themselves answering questions thoroughly and helpfully will be resistant to rejecting questions or aborting the survey. however, keep in mind the effect of recency too   it may be better to place contentious questions near the end but not right at the end so that you don’t leave a bad taste in the respondent’s mouth. most long surveys are not completed. if a questionnaire is long, the person must either be very interested in the topic, an employee, or paid for their time if they’re going to make it through to the end. that means beyond a certain survey length, your questions will not receive equal care and attention from survey takers. how long is too long? the general rule of thumb is to keep the survey short, typically fewer than five minutes. this translates into about 15 simple questions (rather than matrix questions that require many responses in one). the average respondent is able to complete about 3 multiple choice questions per minute. an open ended text response question counts for about three multiple choice questions depending, of course, on the difficulty of the question. while only a rule of thumb, this formula will accurately predict the limits of your survey. as well as keeping questionnaires short and to the point, you can ease the risk of fatigue and drop offs by using the most accessible language you can. you will help ensure your respondents understand your survey if you lower the level of sophistication in your survey writing to the 9 11th grade level. questionnaires often include “ringer or throw away” questions to increase interest and willingness to respond to a survey. these questions are about hot topics of the day and often have little to do with the survey. while these questions will definitely spice up a boring survey, they require valuable space that could be devoted to the main topic of interest. use this type of question with caution. get started with our free survey maker tool surveys surveys surveys surveys surveys surveys surveys step /2 ready to learn more about qualtrics?

your all in one platform for the right digital experience visualize journeys and level up engagement get the whole picture of multi session journeys make better decisions with fast user feedback power up performance and remove friction your go to event for all things customer experience insights to refine your customer experience share tips and tricks with other contentsquare users how well do you understand your users? who are they, what keeps them awake at night, and what brought them to your business in search of a solution? nobody knows more about what your users want than your users themselves. so why not ask them? surveying people directly on your website gives you the feedback you need to improve the user experience (ux) and increase conversions. writing ‘good’ survey questions and knowing when to ask them is paramount to ensuring you get the most out of your survey without wasting your or your users’ time. in this guide chapter, learn about the different types of survey questions, what to ask, where to place surveys on your site, and how contentsquare’s voice of the customer product makes drafting survey questions a breeze it’s all not as intimidating as it might seem, we promise. use ai powered surveys to ask your users the right questions and better understand their needs so you can improve the user experience and boost your business. posing survey questions to your users is the simplest way to understand their pain points, needs, and motivations. but first, you need to know what questions to ask and when to get the answers you need. let’s dive into our list of survey question examples with a breakdown of the six main categories your questions will fall into: open ended closed ended nominal likert scale rating scale yes/no open ended survey questions give your respondents the freedom to answer in depth and in their own words, instead of limiting their response to a set of pre selected choices (such as multiple choice questions, rating scales, or yes and no). to get answers that hold valuable insights, you need to ask engaging and specific questions that invite your visitors to fill that empty text box with relevant customer feedback. what other [product/service] would you like to see us offer? what could we have done better? what led you to make this purchase? what other information would you like to see on this page? was there anything specific that made you [cancel/downgrade]? open ended questions allow for survey responses with depth and context ask open ended questions when you first start surveying and collecting feedback from your customers. if you don't have a great deal of insight yet, answers to open ended questions go a long way in determining who your customers are and what they're looking for. an open ended question is also a good follow up to a closed ended one: asking your users to share more details helps you get an even better understanding of how they feel. do you have a general survey goal in mind but are struggling to determine the right questions to ask? use contentsquare’s voice of the customer to start writing survey questions. choose from our 40+ survey templates, or watch as our ai assistant generates one for you in seconds with questions tailored to the exact purpose of the survey you want to run. easily conduct survey analysis with our ai generated summary reports that track user sentiment and provide an overview of key insights and suggest next steps for you to take based on your survey responses. use contentsquare’s customizable templates or have our ai assistant generate survey questions in seconds closed ended survey questions limit a user’s response options to a set of pre selected multiple choice answers with checkboxes or rating scale answers. this broad category of questions includes nominal likert scale rating scale yes/no we go over each one in the following sections, but let’s linger on the general category for a moment and see some examples. writing your survey questions in a closed ended structure helps you easily quantify feedback when it comes time for data analysis. what’s the main reason for your [downgrade/cancellation]? (list example reasons as answer choices) were you able to find the information you were looking for? (yes/no) how would you rate our customer support on a scale of 0 10? (0 = terrible, 10 = stellar) how likely are you to recommend us to a friend or colleague? (nps® question) how would you feel if we discontinued this feature? (emoji rating scale) while they can be used just about anywhere, closed ended questions work brilliantly in two scenarios: to draw respondents in with a simple opening question, since they require little time and effort and are therefore easy for people to answer. this is called the foot in the door principle: once someone commits to answering the first question, they might be more likely to answer an open ended question that follows. when you need to create graphs and trends based on people’s answers. responses to closed ended questions are easy to quantify and use as benchmarks. rating scale questions in particular allow you to gather customer sentiment and compare your progress over time. a nominal question is a type of closed ended survey question that presents respondents with multiple answer choices; the response options are non numerical in nature and don't overlap (unless you include an ‘all of the above’ option). you can let users choose only one answer, or allow them to select multiple. what are you using our [product/service] for? ✅ business ✅ personal use ✅ other where did you first hear about us? ✅social media ✅word of mouth ✅newspaper article ✅other if you could no longer use our [product/service], what would you miss the most? ➕ (list different features as answer options) what's the next feature or functionality we should build? ➕ (list planned updates from your product roadmap as options) what other content would you like to see us offer? ➕ (list content formats you haven’t used yet as options) nominal questions are perfect for market research and make for good demographic survey questions, when you want context about your respondents and target audience so you can segment them later. nominal data is quantitative and easy to create graphs and trends from, so you can gain a nice overview of the behaviors and attitudes of your different user groups. likert scale questions typically use a 5  or 7 point rating scale that measures someone’s level of intensity to a statement. likert scales are also known as ordinal scales because the answers are presented in a specific order. the scale develops symmetrically: the median number (a 3 on a 5 point scale, for example) indicates a point of neutrality, the lowest number (always 1) indicates an extreme view, and the highest number (a 7 on a 7 point scale, for example) indicates the opposite extreme view. asking likert scale questions is an effective method for turning qualitative data, such as customer sentiment, into quantitative data. this quick customer satisfaction survey contains a 5 point likert scale on a scale of 1 to 7, how satisfied would you feel if we offered this [feature]? not satisfied at all not satisfied somewhat not satisfied neutral somewhat satisfied satisfied very satisfied on a scale of 1 to 5, how much do you agree with this statement: our [product/service] makes [x job to be done] easier. strongly disagree disagree neutral agree strongly agree how would you rate the support you received? not helpful at all not helpful neutral helpful very helpful 💡pro tip: ask different open ended follow up questions depending on the responses you get to your likert scale. ask those who answered 1 3, ‘how could we improve?’ and those who answered 4 5, ‘what did you love about the experience?’ likert scale questions come in handy when you already have some sense of what your customers are thinking. for example, if your open ended questions uncover a complaint about a recent change to your ordering process, you could use a likert scale question to determine how the average user felt about the change. rating scale questions map possible answers on a numeric scale. for example, you can ask shoppers to rate customer support on a scale of 1 5 or the likelihood of them recommending your product on a scale of 0 10 (also known as a net promoter score®). when you use a rating scale question in a survey, make it clear what the scale means (1= poor and 5 = amazing, for example). consider adding a follow up open ended question to understand why the user gave you that score. on a scale of 1 10, how appealing do you find our product offering? on a scale of 1 5, how helpful is the information on this page? how easy was it to use our service? (customer effort score) use a rating scale question to measure nps®, then follow up with an open ended question to gain more context whenever you want to assign a numerical value to your survey or visualize and compare trends, a rating scale question is the way to go. this type of question is often used for customer satisfaction surveys, customer effort score surveys, and product reviews. use contentsquare to set up an nps® survey and trigger it to appear after important events, such as when a user completes a purchase or accesses support. a low nps® alerts you to improvements you need to make, while a high nps® gives you an idea of what to prioritize to boost customer retention. the response format allows you to look at results historically and see if you’re improving or getting worse. binary or dichotomous questions are super straightforward, requiring a simple ‘yes’ or ‘no’ reply. though this question type has the most limiting answer options, it helps you gain a quick sense of user sentiment or the effectiveness of a particular product feature. to make the question even more engaging, try replacing yes/no with 👍and 👎 or 🙂and 🙁. was this article useful? (yes/no) did you find what you were looking for today? (👍/👎) did you have any concerns before buying [product name]? (yes/no) yes/no questions are a good way to quickly segment your respondents. for example, say you’re trying to understand what obstacles or objections prevent people from trying your product. you can place a survey on your pricing page asking people if something’s stopping them, then follow up with the segment who replied ‘yes’ by asking them to elaborate further. a yes/no question also helps you get your user’s attention because it requires very little effort to answer. once they commit to answering that first question, they tend to become more willing to answer questions that follow, or even leave you their contact information. speaking of getting your foot in the door to unlock more insights from your survey respondents, why not ask them to do a follow up interview or usability test with you? contentsquare makes it easy, scalable, and cost efficient to connect with your users: run effortless, automated user interviews and unmoderated usability tests in less than 10 minutes get instant access to a pool of 200,000+ participants from over 130 countries and 25 industries (or invite your own) take notes while contentsquare records and transcribes your interview, then highlight moments that reveal key customer insights and share them with the team identify site errors or validate design concepts, content, and messaging with iterative usability testing to ensure you meet user expectations if you want your users to answer questions and provide feedback that contains actionable insights, you need to ask your survey questions at the right place and time. ask yourself these questions to determine where to place your surveys: what pages are crucial to your business? which over  or under performing pages could most benefit from additional insight? where along the customer journey or conversion funnel have you spotted a leak that needs investigating? 💡pro tip: if you want your users to fill out a somewhat longer questionnaire, you can also send your questions as a link survey via email or post it on social media. be mindful of their time minimize the number of questions or consider incentivizing participation. carefully considering which pages to ask a question on before you start writing one helps improve your chances of getting truly valuable survey data instead of unhelpful answers that won’t lead to improvement. here are five likely candidates for where to ask your survey questions: homepages landing pages success and post purchase pages pages with high exit or bounce rates cancellation, downgrade, or churn pages contentsquare has pre written questions you can place on specific pages, like those with high exit rates to make the most of your survey, you need to know how to write good questions that give you answers you can act upon. we’ve run through a handful of our favorite survey questions but what is it that makes a good survey question, well, good? the goal here is effectiveness: the best survey questions help you get clear insights and business critical information about your customers, including who your target market is how to price your products and services what’s stopping people from buying from you why visitors leave your website with this information, you can tailor your website, products, landing pages, and messaging to improve the user experience and, ultimately, maximize conversions. 💡pro tip: the more attention you put into writing your survey questions, the easier it is to analyze the data you collect. read the dedicated chapter to learn how to conduct survey analysis. focus your questions on the customer’s perspective to understand their needs, drivers, pain points, and barriers tailor the language to your target audience be polite and concise if needed, break things down into separate questions to make for a better survey taking experience consider the foot in the door principle and start your survey with a simple yes/no question, then follow up with an open ended question to increase the response rate don’t ask double barreled questions and stick to one topic per question don’t rely on closed ended questions at the beginning of your research process start with open ended questions to gain an initial impression of your customers and avoid reinforcing your own assumptions avoid asking loaded or leading questions that prompt respondents to answer a certain way for example, rephrase “is our product the best on the market?” to “how does our product compare to other options on the market?” so respondents can answer more freely asking the right questions sets you up to achieve your survey goals it helps you optimize your customer journey, analyze product market fit, increase customer satisfaction, and redefine your messaging. read the dedicated chapter to learn how to create an effective survey with your user and business goals in mind. that said, online surveys aren’t enough in and of themselves to get a comprehensive picture of your customer experience. to gain a richer understanding of your users, combine survey data with insights from tools like session replays, heatmaps, web traffic analytics, or journey mapping. here’s how to do that with contentsquare: discover issues and understand what’s causing them by watching recordings of users who leave negative survey feedback with session replay reveal the behavioral context behind how users interact with your website or product with heatmaps build user segments from your survey responses and follow their user journeys with journeys use ai powered surveys to ask your users the right questions and better understand their needs so you can improve the user experience and boost your business. there are six main survey question types: open ended closed ended nominal likert scale rating scale yes/no homepages landing pages success and post purchase pages high exit or bounce rate pages cancellation, downgrade, or churn pages asking effective survey questions helps you achieve various business goals, such as mapping your customer journey conducting market research and product market fit analysis driving your pricing strategy and understanding why your product isn’t selling gauging customer satisfaction and finding out why people leave or downgrade understanding your customers’ fears and concerns gathering effective testimonials measuring word of mouth recommendations redefining your messaging a ‘good’ question is an effective question: the best ones help you get clear insights and business critical information about your customers, including who your target market is how to price your products/services what’s stopping people from buying from you why visitors leave your website we’re an international team of content experts and writers with a passion for all things customer experience (cx). from best practices to the hottest trends in digital, we’ve got it covered. explore our guides to learn everything you need to know to create experiences that your customers will love. happy reading! © 2025 contentsquare. all rights reserved.

products surveymonkey is built to handle every use case and need. explore our product to learn how surveymonkey can work for you. get data driven insights from a global leader in online surveys. explore core features and advanced tools in one powerful platform. build and customize online forms to collect info and payments. integrate with 100+ apps and plug ins to get more done. purpose built solutions for all of your market research needs. create better surveys and spot insights quickly with built in ai. contact sales templates measure customer satisfaction and loyalty for your business. learn what makes customers happy and turn them into advocates. get actionable insights to improve the user experience. collect contact information from prospects, invitees, and more. easily collect and track rsvps for your next event. find out what attendees want so that you can improve your next event. uncover insights to boost engagement and drive better results. get feedback from your attendees so you can run better meetings. use peer feedback to help improve employee performance. create better courses and improve teaching methods. learn how students rate the course material and its presentation. find out what your customers think about your new product ideas. contact sales pricing enterprise resources best practices for using surveys and survey data our blog about surveys, tips for business, and more. tutorials and how to guides for using surveymonkey. how top brands drive growth with surveymonkey. contact sales products surveymonkey is built to handle every use case and need. explore our product to learn how surveymonkey can work for you. get data driven insights from a global leader in online surveys. explore core features and advanced tools in one powerful platform. build and customize online forms to collect info and payments. integrate with 100+ apps and plug ins to get more done. purpose built solutions for all of your market research needs. create better surveys and spot insights quickly with built in ai. templates measure customer satisfaction and loyalty for your business. learn what makes customers happy and turn them into advocates. get actionable insights to improve the user experience. collect contact information from prospects, invitees, and more. easily collect and track rsvps for your next event. find out what attendees want so that you can improve your next event. uncover insights to boost engagement and drive better results. get feedback from your attendees so you can run better meetings. use peer feedback to help improve employee performance. create better courses and improve teaching methods. learn how students rate the course material and its presentation. find out what your customers think about your new product ideas. pricing enterprise resources best practices for using surveys and survey data our blog about surveys, tips for business, and more. tutorials and how to guides for using surveymonkey. how top brands drive growth with surveymonkey. effective surveys start with asking the right questions. understanding the different types of survey questions ensures you’re gathering the most insightful, actionable data from your respondents. whether you're conducting academic research, improving customer satisfaction, or boosting employee engagement, creating a well designed survey is key to gathering actionable insights. the types of questions you ask in a survey play a crucial role in collecting accurate, relevant data. different question types can yield varying levels of detail, accuracy, and engagement from respondents. in this article, we'll explore common types of survey questions, from multiple choice to likert scale, and offer examples and expert tips on how to use each for the best results. by understanding the nuances of these question formats, you can design surveys that not only capture the data you need but also enhance the overall respondent experience. creating an effective survey involves selecting the right types of questions to gather the data you need. below is a snapshot of the most commonly used survey question types and when to use them. keep reading for a more in depth exploration of the most common types of survey questions and how they can be effectively utilized to gather valuable insights. multiple choice questions are the most popular survey question type. they allow your respondents to select one or more options from a list of answers you define. they’re intuitive, help produce easy to analyze data, and provide mutually exclusive choices. because the answer options are fixed, your respondents have a more effortless survey taking experience. perhaps, most importantly, you’ll get structured survey responses that produce clean data for analysis. multiple choice questions come in many different formats. the most basic variation is the single answer multiple choice question. single answer questions use a radio button format (circle buttons representing options in a list) to allow respondents to click only one answer. they work well for binary questions, questions with ratings, or nominal scales. multiple answer multiple choice questions are commonly shown with square checkboxes. they allow respondents to check off all the choices that apply to them. for example, "in which of the following ways do you use our product?" a common drawback of multiple choice questions is that they force you to limit responses to a predetermined list of options, which can cause bias in your results. what if none of your answer options apply to your respondents? they might just choose a random answer, which could impact the accuracy of your results. you can solve this problem by adding an “other” answer option or comment field. it should be listed at the end of all your choices. when respondents see it, they know they have the option of answering your question in their own words instead of yours. in rating scale questions (sometimes referred to as ordinal questions), the question displays a scale of answer options from any range (0 to 100, 1 to 10, etc.). the respondent selects the number that most accurately represents their response. net promoter score® questions are a good example of rating scale questions. they use a scale to gauge how likely customers would be to recommend their product or service. with ranking questions and numerical rating scales it’s important to give the respondent context. for instance, imagine you asked the question, “how much do you like ice cream?” without explaining the value of the numbers on your scale, a numerical rating scale might not make much sense. chances are you’ve seen this question type before. likert scale questions are the “do you agree or disagree” questions you often see in surveys and are used to gauge respondents’ opinions and feelings. likert scale questions give respondents a range of options for example, starting at “not at all likely” and scaling up to “extremely likely.” that’s why they work well to understand specific feedback. for example, survey questions for employees often use a likert scale to measure their opinions or attitudes on a range of topics. related reading: likert scale survey questions, examples, & mistakes to avoid if you want to ask a few questions in a row that have the same response options, matrix questions are your best option. a series of likert scale questions or a series of rating scale questions can work well as a matrix question. matrix questions can simplify a lot of content, but it’s important to use them carefully. very large matrices, like the one below, can be confusing and difficult to take on mobile devices. if the matrix becomes too complex or lengthy, consider breaking it into smaller, more manageable sections or alternative question formats, such as dropdown menus or multiple choice questions. the dropdown question is an easy way to display a long list of multiple choice answers without overwhelming your respondents. with it, you can give them a scrollable list of answers to choose from. sometimes, showing all answer options at once can offer your respondents useful context about the question. keep that in mind whenever you consider using more than one dropdown question in your survey. open ended survey questions require respondents to type their answers into a comment box and don’t provide specific pre set answer options. responses are then viewed individually or by text analysis tools. when it comes to analyzing data, open ended questions aren’t the best option. it’s not easy to quantify written answers which is why text boxes are better for providing qualitative data. allowing your respondents to offer feedback in their own words could help you uncover opportunities that you may have otherwise overlooked. however, if you’re looking for data to analyze, you may want to engage in quantitative marketing research and utilize closed questions. pair closed ended questions with open ended ones to better understand and address your quantitative data. for example, after the net promoter score survey question, you can ask respondents the reason behind their score: use demographic survey questions if you’re interested in gathering information about a respondent’s background (e.g., income level, age, location). when properly used, these types of questions in a questionnaire allow you to gain better insights into your target audience. demographic questions are powerful tools for segmenting your audience based on who they are and what they do, allowing you to take an even deeper dive into your data. typical demographic questions ask for information like age, gender, relationship status, and occupation. a ranking question asks respondents to order answer choices by way of preference. this allows you to not only understand how respondents feel about each answer option but also helps you understand each one’s relative popularity. it’s important to keep in mind that ranking questions can take more time to answer. so try not to use them if other question types can provide the data you need. also, only use them when you’re confident that respondents are familiar with each answer option. otherwise, they’ll be next to impossible to answer accurately and honestly. for instance, in the question below, respondents need to be familiar with each tv show before they can compare them. our image choice question type allows you to use images as answer options. this works great when you want respondents to evaluate the visual qualities of something, such as an ad or a logo. want to get real time, gut reaction feedback on an image? use a click map question! add an image to your survey and ask survey takers to click a certain spot on the image. for example, you could ask what item on a shelf is most appealing or which part of your website is most user friendly. need respondents to upload their resume? a headshot? their id? you can collect whatever you need as a pdf, png, or doc file. and once your responses come back, you can easily download the files. with a slider question, respondents can evaluate something on a numerical scale. the questions are interactive, which can make them fun to answer, and they allow you to quantify respondent sentiment at both an individual and aggregate level. benchmarkable questions aren’t necessarily presented in a specific format, but they’re special in that they allow you to compare yourself to other survey creators who asked the same question. benchmarkable questions range widely, and they can be used for different audiences (employees, customers, etc.). to find one for your survey, you can either use our question bank or one of our survey templates. any question that has a small bar chart icon at the top means it’s benchmarkable. dichotomous questions are super straightforward they give you just two choices, usually "yes" or "no," or picking between two different things. these questions are characterized by their binary nature, offering respondents a clear and simple choice between two options. for example: qualitative questions focus on gathering descriptive or narrative information from participants. they seek stories, opinions, or explanations directly from the source. open ended questions are the most commonly used qualitative question types in surveys. the qualitative data collected from these questions is typically textual and non numerical. for example, you might ask your customers, "what do you think about our new product?" or "can you describe your experience with our customer service team?" qualitative questions are useful for exploring issues that cannot be easily summarized with numbers alone or for tapping into people's feelings, attitudes, and perceptions. however, qualitative data cannot be easily quantified and are often context specific, meaning they may not be representative of broader populations or trends. quantitative questions focus on numbers and measurements. they provide options for responses, such as rating scales or multiple choice questions. quantitative data, consisting of numerical values, can be statistically analyzed. examples of quantitative questions include: when you want to assess specific attitudes or perceptions in a standardized and measurable way, quantitative questions allow you to collect numerical data that can be analyzed statistically. they are ideal for measuring the frequency, intensity, or magnitude of behaviors, experiences, or events. however, quantitative questions typically provide predefined response options or scales, which can limit respondents' ability to express nuanced or complex opinions, potentially leading to a shallower understanding of respondents' perspectives compared to qualitative questions. now that you know the different survey questions examples, you’re ready to start creating your survey. here’s some advice to keep in mind as you begin: creating a great survey doesn’t have to be difficult. leverage our hundreds of free, expert designed survey templates built by in house subject matter experts and survey scientists. if you prefer to create a survey from scratch, use our question bank to start populating your survey with ready made questions that can help you get the answers you need. this ensures your survey is methodologically sound and yields unbiased results. are your surveys ready to go mobile? approximately 3 in 10 people taking surveymonkey surveys in the u.s. do so on a smartphone or tablet. here’s a quick checklist to ensure your survey is mobile optimized: avoid jargon, technical terms, or complex language that might confuse respondents. ensure each question is clear and focused on one topic at a time to avoid ambiguity. craft questions that do not lead or influence respondents towards a particular answer. instead of asking, "how great was your experience with our customer service?" ask, "how would you rate your experience with our customer service?" sprinkle in some open ended questions alongside your numerical ones. such a combination allows respondents to provide additional context or elaborate on their responses. this can uncover valuable insights and enrich your data. we can all benefit from someone proofreading our work. surveys are no exception. share your survey with colleagues, friends, family, and anyone else who has a vested interest in its success before you send it. who knows? they might be able to help you spot errors or areas for improvement! it is important to know which survey question type to use and when to leverage it. once you’re familiar with different survey questions, you’ll be able to focus on what you need from respondents, resulting in far better data. before launching your survey, pretest questions with a small sample of respondents to identify any issues with wording, comprehension, or response options. use feedback to refine and improve your questions. related: sample survey questions and examples ready to level up your survey game? dive into our template gallery, where we've curated a collection of expertly designed surveys and survey questions to help you gather valuable insights and make informed decisions. whether you're exploring customer satisfaction, employee engagement, or market research, we've got you covered with a variety of templates to suit your needs. learn survey design fundamentals, ai powered creation, and collaboration best practices. nps, net promoter & net promoter score are registered trademarks of satmetrix systems, inc., bain & company and fred reichheld. explore our customer satisfaction survey templates to rapidly collect data, identify pain points, and improve your customer experience. discover how ryanair uses surveymonkey and its microsoft power bi integration to track 500k monthly csat surveys and improve customer experiences. is your company meeting customer expectations? use our free csat calculator to assess your customer satisfaction score and drive profits. read our step by step guide on conducting customer behavior analysis. learn how to collect data and improve customer touchpoints. product popular templates resources learn about
by ruth igielnik on top of all the other challenges that pollsters have faced in the past two presidential elections, this year has an additional, potentially significant, complication: a well known third party candidate. measuring support for third party candidates has long been a particular challenge for pollsters. but it has been decades since the country has seen a third party candidate as prominent as robert f. kennedy jr., who has an average of about 10 percent of the vote in national polls. historically, polls overstate support for third party candidates. when it comes to mr. kennedy, the biggest question may be by how much. kenny holston/the new york times consider this: in a two part experiment conducted by the new york times and the research firm ipsos, a seemingly subtle difference across two versions yielded significantly different results for mr. kennedy. what’s more, a candidate who is not on the ballot anywhere   a times editor picked for inclusion thanks to his generic sounding name   received a non negligible share of support, highlighting just how much support for third party candidates can come down to frustration with major party candidates and a yearning for more options. what does this all mean? it’s tempting to interpret these results as ungenerous to both voters and to polling. but the results say something real about how preferences work, and the central conundrum when it comes to third party candidates. (this experiment is separate from times/siena college polling, though it was conducted with the same standards and rigor that we apply to all of our polling.) in short: much of what influences third party candidate support isn’t just a straightforward desire to see that person become president. this poses a challenge for pollsters no matter what they do: simply by listing third party candidates, a poll might overstate their support. if a poll doesn’t list them, however, it can’t capture their support at all. this year, to combat that concern, many reputable pollsters ask both versions of the question: one that poses a simple head to head contest between major party candidates, and one that includes third party candidates who may be on the ballot. and which question gets asked first is where the difference comes in. here is the longer question asked by times/ipsos that includes the full field: long list thinking about the presidential election in november, if the election were held today, who would you vote for if the candidates were … [1] joe biden, the democrat [2] donald trump, the republican [3] robert f. kennedy jr., the third party candidate [4] jill stein, the green party candidate [5] lars mapstead, the libertarian party candidate [6] william davis, the third party candidate thinking about the presidential election in november, if the election were held today, who would you vote for if the candidates were … [1] joe biden, the democrat [2] donald trump, the republican