CURRENT | ARCHIVES

Print Email to a Friend

Sept. 1, 2005

Testing your political knowledge? Correct answers may involve time, money

ANN ARBOR, Mich.—Survey participants who are stumped by questions about politics may do better with a little more time, and perhaps some money, a new study shows.

A national study co-authored by University of Michigan researcher Arthur Lupia found that when people are called on the phone at home with no prior warning and asked for immediate responses to survey questions, they are less likely to know the right answer than if they have additional time or incentives.

The paper, written by Lupia and Markus Prior of Princeton University, will be presented Friday (Sept. 2) at annual American Political Science Association conference in Washington D.C.

“Survey respondents may perform poorly on political knowledge tests not because they are incapable of answering the questions, but because they are caught unprepared and unmotivated to perform well,” said Lupia, a political science professor and researcher at the Institute for Social Research.

Extra time and a small monetary incentive increased the number of correct answers by at least 10 percent, the research showed.

In many circumstances, political scientists and other observers care about what people know about politics as an important decision, such as a vote on election day, approaches. What people know when asked political questions out of the blue by a stranger is less relevant than what people know about politics and policy after they talk to friends or have an opportunity to seek information themselves. Thus, the answers that people give in most survey interviews – which do not allow respondents to seek such information and provide little motivation for answering questions correctly – are unlike the answers they might give on Election Day, the study shows.

A national sample of more than 1,200 people was asked 14 questions about politics, economics and public policy. Some questions pertained to the 2004 presidential elections. Half of the sample was given only 1 minute to answer each question, while the other half was given up to 24 hours to complete the entire set of questions.

 In addition, half of the sample received a small monetary reward for each question answered correctly ($1 for a correct answer) while others received nothing. So, roughly one-quarter of the respondents were assigned to the “one minute per question, no pay for correct answers” condition—the same conditions presented to citizens in most professional surveys. The other three-quarters were given either extra time or extra money.

The researchers said their study had two implications. First, those who read political polls should be careful about drawing broad conclusions about citizens’ political knowledge. In many cases, society encourages people to pick up a book or consult with trusted others when making an important decision, and on-the-spot political surveys prevent this, Lupia said.

 
 Some factors may affect how people completed the questionnaire. Respondents with a full-time job or with young children may be disadvantaged in the one-minute condition because demands on their time keep them from regularly following the news, whereas 24-hour conditions allow for greater flexibility, Lupia said.

Pay conditions could motivate low-income respondents to perform well. Income could matter more in the 24-hour conditions because higher-income respondents can easily afford resources such as cable television or newspaper subscriptions.

Some questions (asked Oct. 19 to Nov. 1, 2004), with the correct answers:

• John Kerry said he would eliminate the Bush tax cuts on families making how much money? More than $200,000 a year. (Multiple choice question)
• What percentage of Americans was unemployed in August 2004? Around 5 percent. (Multiple choice)
• Of the 100 members of the U.S. Senate, how many are members of the Republican Party? 51. (Open-ended question).
• What percentage of Americans did not have health insurance in 2003? 15.6, although accepted range was 12.5 to 18.6 (Open-ended).

Related links:

Lupia

APSA

Contact: Jared Wadley
Phone: (734) 936-7819