Political Science professor Christopher Achen, who worked as an ABC analyst on election night, explains how every network got it wrong — twice — and suggests the changes he would love to see in 2004.

The renowned political scientist Christopher Achen spent his undergraduate days at UC-Berkeley studying political science, and went on to get his PhD at Yale, specializing in the relatively new field of quantitative methods. He taught at Berkeley for 10 years, the University of Chicago for six, and is currently in his eleventh year at U-M.

Professor Achen recently took a break to chat with us about his involvement as an ABC analyst in this year's presidential election. After being right in the middle of the action on election night, he offered us an explanation of what went on behind the scenes on election night, and what he hopes will never happen again.

Michigan Review: How can networks project a winner the moment the polls close? How does the system work?

Christopher Achen: When they project, at the time the polls close, they are projecting just on the basis of exit polls. Sometimes exit polls plus sample precincts. But more commonly, just exit polls, so they have not seen a single real vote when they call it at or before the polls close.

MR: How do you guys call it?

In principle, it’s a straightforward statistical problem. You interview people leaving the polls, you interview a sample of perhaps 2,000 or 3,000 people in a typical state - sometimes more in a big state, a very diverse state. That gives you an estimate of what the vote proportion was for the two candidates. If that difference is big enough, and if the sample is big enough, then it’s just like a national public opinion poll, and you can be certain up to a certain margin of error, with calculations that are familiar from statistical textbooks, of how likely that margin is to be correct. The problem is that some people don’t like to talk to the exit pollsters, and that means it’s not really a textbook problem, and so the calculation that they sometimes use isn’t very accurate and that’s what gets them into trouble.

MR: Why are people that vote Democratic more likely to answer exit polls than Republicans? 

That’s a very hard question. I studied this at one point over a weekend on a little contract from VNS three years ago with a couple of other people. The problem appears to be that more educated people are more likely to answer the exit polls. And in particular, people with education beyond the college level, are a great deal more likely to answer exit polls than the other people. Those people tend to vote democratic, and that over represents them. So it’s not a Democratic bias at all, but it’s real.

MR: You mentioned the Voter News Service. What exactly do they do?

They are the people who conduct the interviews on the exit poll, do the sampling of sample precincts, and also keep track of the actual vote as it comes in, all in one central location in New York - namely, the World Trade Center. All the networks, plus the Associated Press, all get exactly the same information. They are the clients for VNS, they pay for it, and in return for that money their employees can sit in front of a computer screen that night and look at all of the data that VNS has.

MR: Is the system effective?

Yes and no. It works very well a lot of the time. It enables the networks to call the states that are one-sided. So they were able to call Utah for George Bush, for example, and the District of Columbia, and Vermont, for Gore, because the exit polls are so clear in those cases that they get the right answer early. In the close states, if you’re patient enough, they will give you the right answer also. But the networks are not always patient.

MR: As we saw. Would you change the system in any way? If you would change it, what would you do differently?

I’d certainly like to change it. It’s a very complicated operation, and I think we ought to go slow and think carefully about how we change it. It would be easy to make it worse. The current system does a tremendously good job at collecting the data - and this is a gigantic task, as you can easily imagine. There are more than three thousand American counties. They [VNS] get the data in quickly, they do a good job collating it with a minimum of errors. That part of the operation is just stellar, and I think they have a lot to be proud of. The part that needs changing is the part that does the estimates, that constructs the information from which it would call the states. It’s simply not very accurate. Sometimes it says 50% chance of Gore winning when it means 90%. Sometimes it says 90% when it means 99%. Sometimes it says 99% when it means 90% or 85%. Just the technical fix is the first thing that needs to be done.

MR: Why isn’t it accurate a lot of times? Because of the people who don’t answer the polls?

A combination of that, and the fact that there are inevitable data errors on election night. I’m a Montanan originally. If you think about voting in the eastern part of Montana, there are people going to the polls there, often in terrible weather, who are a hundred miles from a town of 5,000. That happens a lot, across the country, and you have to get the votes in from all of those places. They’re called in, in most instances, to the secretary of state, and people are tired, they’ve been working long hours, and they make mistakes. That’s just inevitable.

MR: Can you and your people who do the analyzing correct for some of their errors?

Some. When you’re looking, for example at a black precinct and it comes in 50-50 Democrat and Republican, you know that a mistake has been made, and you check. So some things can be caught. In ’96 there were two people running for the Senate in Virginia, both named Warner. That caused problems, as you could imagine.

But as I say, I think the job that VNS does on that part of the task is really tremendous. The statistical software partly makes errors due to the exit poll problem we’ve talked about, and partly it’s very conservative, statistically, in the way that it makes forecasts, and it doesn’t take very good account of the political relationships among states. For example, the software does not know that Ohio and Michigan are next to each other, and that Ohio is always a little more Republican than Michigan - it doesn’t know any of that. So it’s slow, and the combination of too much certitude on the exit polls and not enough certitude on anything else means that it’s erratic, and sometimes too high on its probabilities and sometimes too low.

MR: Let’s get to specifics. How on earth could all the networks call Florida incorrectly at 8 p.m. for Gore, and then incorrectly again at 2:17 a.m. for Bush? Do they ever consider that an incorrect guess might be devastating to the candidates? How’d it happen?

The call at 8 o’clock was a good example of the software saying that it was 99% likely that Gore was going to win the state. The real number, if you’re experienced with the system, was probably 85% or 90%. So on the decision team at ABC, there was a division of opinion about whether it should be called or not. The other networks began calling it; we were the last, because of this disagreement. At 2 o’clock, exactly the same thing happened. There was evidence that it might go for Bush, but again, the estimate was noisy, there was disagreement on