As big scandals go, it lacks big sexiness. A lawsuit between the lefty blog DailyKos and their (former) polling firm Research 2000 has all the thrilling intrigue of a fishing show and paint drying combined. Even my eyes glazed over reading the posts about it at FiveThirtyEight.com -- and I love this factual data stuff.
Ok, so that's how you write an exciting lede. I've totally got you hooked with the promise of crushing boredom, statistics, math, and general inside-baseball wonkiness. Who could possibly resist reading further? Except I'm not going to write about any of that -- it's better covered in the links above than anything I could do. Suffice it to say that Research 200 is probably pretty bad and that DailyKos was more than likely getting ripped off.
What I want to talk about is how dependent we've become on polling. Public opinion polling is a weird thing, both factual and invented, both real and unreal. It's been a long-standing complaint (especially on the right, where their ideas tend to poll badly) that you can get any poll results you want by the phrasing of the questions. That's only kinda sorta true. Polling outfits don't really hide their questions, so there's no reason to assume the question is faulty. "If the election were held today, would you vote for X or Y?" is pretty cut and dried. And that's the way most questions are formed. When people make this argument, you'll notice they never explain what was wrong with the question.
More likely is that the pollster is being selective about their respondents or just making stuff up. If you need to explain an outlier poll, that's where Occam's razor would shave; it's a lot easier to just invent numbers or choose respondents most likely to give the results you want than it is to dig into the psyches of respondents and write a sort of mind-control survey. Want a poll to lean right? Poll mostly Orange County, California and rural Texas or, as Rasmussen does, poll only people who've been voting for a while. Either that or just make crap up.
The problem with bad polling is the bandwagon effect -- polls influence polls. A rational person would look at a poll that shows he or she holds a fringe position and would reconsider their own position. That's not an illogical move. When most people tell you you're wrong, it's not crazy to start thinking that maybe you're wrong. Rule out ideologues and I think we can call this the default position of anyone on anything -- this is what I believe, but I'm willing to be convinced I'm wrong. I can't think of a more rational approach to a question.
But this position doesn't consider that everyone else is operating under bad info. Worse, it doesn't consider the possibility that the group presented as the majority isn't the majority at all. Our opinions may be influenced by a misinformed majority or even a phantom majority. I don't think we can help but be influenced by majority opinion. And I don't think that's wrong or unreasonable or illogical. But it turns out that we now have to question what the majority opinion really is.
Which is really, really bad, to put it very simply. We're social animals. We cry out when we're startled to alert others to possible danger. If we see something on fire, our first impulse to to grab a buddy and say, "Holy crap, that thing's on fire!" The first impulse in not to try to put it out ourselves. We always seek out help in an emergency. In most situations, group-think is not a bad thing; it's evolution in action and one of our survival traits.
But group-think is also prone to propaganda. The majority wanted to invade Iraq and the majority was tragically wrong. We were given bad information designed to influence opinion and that screwed us. I guess what I'm taking too long to say is that we should consider the majority opinion, but we should also look at what influences it. Be aware of the people around you, but not at the expense of being smart. Polling can tell you a lot of things, but it doesn't necessarily tell you the truth. There was a time in human history where the question, "Do you believe the world is flat?" would've "destroyed" the argument that the world was round.
Polling, when it's done right and honestly, is factual data. But that doesn't mean the majority is correct.
-Wisco
Get updates via Twitter
Ok, so that's how you write an exciting lede. I've totally got you hooked with the promise of crushing boredom, statistics, math, and general inside-baseball wonkiness. Who could possibly resist reading further? Except I'm not going to write about any of that -- it's better covered in the links above than anything I could do. Suffice it to say that Research 200 is probably pretty bad and that DailyKos was more than likely getting ripped off.
What I want to talk about is how dependent we've become on polling. Public opinion polling is a weird thing, both factual and invented, both real and unreal. It's been a long-standing complaint (especially on the right, where their ideas tend to poll badly) that you can get any poll results you want by the phrasing of the questions. That's only kinda sorta true. Polling outfits don't really hide their questions, so there's no reason to assume the question is faulty. "If the election were held today, would you vote for X or Y?" is pretty cut and dried. And that's the way most questions are formed. When people make this argument, you'll notice they never explain what was wrong with the question.
More likely is that the pollster is being selective about their respondents or just making stuff up. If you need to explain an outlier poll, that's where Occam's razor would shave; it's a lot easier to just invent numbers or choose respondents most likely to give the results you want than it is to dig into the psyches of respondents and write a sort of mind-control survey. Want a poll to lean right? Poll mostly Orange County, California and rural Texas or, as Rasmussen does, poll only people who've been voting for a while. Either that or just make crap up.
The problem with bad polling is the bandwagon effect -- polls influence polls. A rational person would look at a poll that shows he or she holds a fringe position and would reconsider their own position. That's not an illogical move. When most people tell you you're wrong, it's not crazy to start thinking that maybe you're wrong. Rule out ideologues and I think we can call this the default position of anyone on anything -- this is what I believe, but I'm willing to be convinced I'm wrong. I can't think of a more rational approach to a question.
But this position doesn't consider that everyone else is operating under bad info. Worse, it doesn't consider the possibility that the group presented as the majority isn't the majority at all. Our opinions may be influenced by a misinformed majority or even a phantom majority. I don't think we can help but be influenced by majority opinion. And I don't think that's wrong or unreasonable or illogical. But it turns out that we now have to question what the majority opinion really is.
Which is really, really bad, to put it very simply. We're social animals. We cry out when we're startled to alert others to possible danger. If we see something on fire, our first impulse to to grab a buddy and say, "Holy crap, that thing's on fire!" The first impulse in not to try to put it out ourselves. We always seek out help in an emergency. In most situations, group-think is not a bad thing; it's evolution in action and one of our survival traits.
But group-think is also prone to propaganda. The majority wanted to invade Iraq and the majority was tragically wrong. We were given bad information designed to influence opinion and that screwed us. I guess what I'm taking too long to say is that we should consider the majority opinion, but we should also look at what influences it. Be aware of the people around you, but not at the expense of being smart. Polling can tell you a lot of things, but it doesn't necessarily tell you the truth. There was a time in human history where the question, "Do you believe the world is flat?" would've "destroyed" the argument that the world was round.
Polling, when it's done right and honestly, is factual data. But that doesn't mean the majority is correct.
-Wisco
Get updates via Twitter
