Norbert Schwarz, professor of psychology, ISR senior research scientist

"My favorite finding is this: we did a study where we asked students, 'How satisfied are you with your life? How often do you have a date?' The two answers were not statistically related-you would conclude that there is no relationship between dating frequency and life satisfaction. But when we reversed the order and asked, 'How often do you have a date? How satisfied are you with your life?' the statistical relationship was a strong one. You would now conclude that there is nothing as important in a student's life as dating frequency.

Photo by Marcia L. Ledford, U-M Photo Services Schwarz

"This is an example of a question-order effect—which is just one of many ways that survey questions can affect the answers we receive. Most questions do not influence one another. But when questions are substantively related, then question-order effects like this one are among the most stable and reliable findings in the survey research literature.

"We wanted to look at what happens to question-order effects as people age, because as we age, our memory gets worse, and as memory gets worse, we figured that questions asked earlier in the survey wouldn't have as much influence.

"We found that question-order effects disappear as people get older. Some people said this finding didn't mean much. They said it could just reflect the fact that when we're old, we've finally figured out what our views are, and our opinions aren't influenced as much by context.

"Now, it's not clear why it would take 60 years to figure out what you believe! But to see if that was the case or not, we did follow-up studies in the laboratory to compare older people whose working memories were very good, older people whose working memories were not so good, and younger people. And we found that older adults with good working memories still showed the same question-order effects as younger adults, whereas for older adults with poor working memories, question-order effects pretty much disappear.

"We also looked at the effect of age on another well-established way that questions affect answers: response-order effects. By and large people are more likely to agree with the last choice mentioned in a phone interview, but the first choice presented in a written questionnaire. If older adults have finally figured out what they believe, in contrast to the rest of us, you should see response-order effects also decrease with age, just like question-order effects do. But that's not what we found. Response order effects go up with age. What memory loss does is attenuate question-order effects while it increases response-order effects.

"So when people cite various survey findings to show that older people are more conservative, say, than younger people, some of the conclusions may to some extent be a result of the way the questions were asked. To see how question-order and response-order influence the findings, you should split the sample so you ask the question one way to one group, another way to another group. Then you would be able to get a better handle on this problem and adjust for these effects in your analyses of the results. But many surveys do not do this."

How Age Influences Answers

"Do you think it should be possible for a pregnant woman to obtain a legal abortion if she is married and does not want any more children?" (Question A)

"Do you think it should be possible for a pregnant woman to obtain a legal abortion if there is a strong chance of serious defect in the baby?" (Question B)

In 1979, U-M researcher Howard Schuman asked these two questions as part of the Detroit Area Study, with half the 777 respondents asked in A-B order and half in B-A order. He found that support for abortion because a woman "does not want any more children" (A) was higher when this question was asked first and dropped dramatically when preceded by the child defect question (B). "This experiment has been repeated many times," notes psychologist Norbert Schwarz. "Although the absolute numbers vary, the pattern remains the same."

Schwarz and colleagues Bärbel Knäuper at McGill University in Montreal and Denise Park at the U-M recently re-analyzed these data and found that the question-order effect decreased dramatically with respondent age. While 69 percent of respondents age 18-54 supported abortion in the case of a woman who "does not want any more children" when Question A was asked first, only 50 percent did so when Question B was asked first, as expected. For respondents age 65 and older, however, question order made no difference.

In a survey on divorce, age had the opposite effect on response-order effects, Schwarz and his colleagues found. Older respondents were no more likely than younger people to say that divorce should be more difficult to obtain when "more difficult" was presented as the middle of three answer choices. But when "more difficult" was the last choice presented, older people were much more likely than younger respondents to endorse this choice as the answer.