Search Archives:

Custom Search

Tuesday, March 04, 2008

Statistical Soup

It's not a prestigious poll or a poll conducted by a giant cable news network. But it's a typical poll. An Austin, Texas TV channel finds pretty much nothing.

KVUE’s latest Belo Texas poll shows voter turnout will be key for Senators Hillary Clinton and Barack Obama -- the democratic winner is anyone’s guess.

Voters decide who they prefer for president Tuesday and according to the latest poll, Clinton lead Obama by just one point -- 46 percent to 45 percent -- both numbers within the margin of error.

Neck and neck, anyone's game, a real nailbiter.

Except that's not how most news outlets are reporting polls. KVUE mentions the margin of error in the second paragraph -- most news outlets don't mention them at all. Other outlets tell you that Clinton's ahead in one poll, behind in another, while the margin of error renders these assessments meaningless.

The worst offender may be CNN, with a statistical monstrosity they call "the poll of polls." They jam together entirely different polls asking entirely different questions of entirely different demographics selected in entirely different ways for entirely different reasons and present this mathematical mishmash as if it meant a damned thing.

CNN's poll of polls, an averaging of the most recent surveys in each state, suggests the race is extremely tight, with Obama ahead by 2 points in Texas and Clinton ahead by 5 in Ohio. But the polls also indicate there are still many undecided voters in both states.

Of course, this is CNN going out of its way to avoid telling you that they don't have any useful information for you. "We Don't Know" is never a headline.

I'll admit it and it's no surprise to anyone who's read me more than once -- I'm a political junkie. I watch polls the way farmers watch the Weather Channel. And, like those farmers, I know the difference between forecast and prediction. Polls aren't predictive -- as much as they're presented that way. They're merely factual data that tell truth about a moment.

So, left with polling results that say, "We Don't Know," news outlets go out of their way to avoid that headline. Clinton's up here, Obama's up there, and if we combine several numbers that are related only by subject, we see that they add up to jack -- let's see if we can pull a story out of that.

The fact of the matter is that, in Texas at least, the race is just too close to call. That's the fact and that's the story -- as less than compelling as that story is. Networks don't like doing this the old-fashioned way. They don't like having to wait until the polls close to report the winner. If pre-polling won't do the job, then exit polling will.

The problem here is that networks keep up the pretense of waiting until the polls close to project a winner. In the Wisconsin primary, CNN projected McCain the winner -- ten minutes after the polls closed and with 0% of precincts reporting. OK, it wasn't really going out on a limb, but you get what I'm saying here.

In New Hampshire, this idea of the predictive poll came back to bite the media in the ass. And, faced with having to explain why they practically crowned Barack Obama the winner, they changed the narrative. "Why were the polls wrong?" everyone asked.

But the polls weren't wrong, they just weren't predictive. And, looking at them now, it's hard to see how anyone would've thought they could be. Pollster John Zogby goes a long way toward explaining what happened:

According to the exit polls, 18% of the voters said that they made up their minds on primary day. That is just an unprecedented number. I have polled many races, especially close ones, where 4% to 8% have said they finally decided on their vote the day of the election and that can wreak havoc on those of us who are in the business of capturing pre-election movements and trends. But nearly one in five this time?

Add the undecided to the margin of error in New Hampshire pre-polling and you've got a pretty substantial number. To look at data, see that big percentage of grey fog, and say you can see the future is either foolish or insane. The polls were correct, they just weren't very useful. When you have 18% making their decision at the last minute and polls showing a single-digit difference among the contenders, then you've got nothing, polling-wise.

You may be eager to know how the primaries and caucuses turn out today -- I know I am. For one thing, the future of Hillary Clinton's campaign may hinge on the results. But the polls are meaningless today and you're just going to have to wait. KVUE has it right -- nobody knows.

And that's the real story, because that's all the polls are saying.


Technorati tags: ; ; ; ; ; ; ; ; ; ; ; The hates it when s don't tell them jack