Damned Lies and Statistics

by: Thurman Hart

Mon Apr 24, 2006 at 06:33:54 PM EDT

It's screwed.

It's too easy to run a poll and too easy to run a bad one that tells you nothing. 

Here are the results of the three lates polls on the NJ Senate race:

Poll	Kean	Menendez
Public Mind	42	38
Rasumssen	43	35
Strategic Vision	34	32

So which one do you believe?

Public Mind says:

The PublicMind poll of 685 randomly selected registered voters statewide was conducted by telephone from March 27 through April 2 and has a margin of error of +/- 4 percentage points.

Rasmussen says:

The telephone survey of 500 Likely Voters was conducted by Rasmussen Reports March 28, 2006.  The margin of sampling error for the survey is +/- 4.5 percentage points at the midpoint with a 95% level of confidence.

Strategic Vision says:

Telephone interviews with 800 likely New Jersey voters, conducted from Apr. 7 to Apr. 9, 2006. Margin of error is 3 per cent.

Public Mind looks at "registered voters".  The only problem is that most registered voters are very likely to stay home in November.  In other words, it's a totally useless poll.

Rasmussen and Strategic Vision both poll "likely voters", which is generally determined by asking someone if they are likely to vote.  It isn't perfect because, believe it or not, people lie.  However, it's the only way to try and weed things down a bit.

So, which one of those do you believe?  Answer: It doesn't matter.  First, they are both picking the same leader: Tom Kean, Jr.  Second, they are both indicating that a very large part of the population has no idea who they want to vote for. For Rasmussen, twice the difference between the two choices hasn't decided.  For Strategic Vision, "undecideds" make up as large a group as either that show a preference.

Having said that, there are significant differences between the two.  Rasmussen has Kean ahead by more than the Margin of Error.  Strategic Vision basically has them deadlocked.

Here's a rule of thumb for polls - the more people they poll, the closer to reality they are likely to get.  If you just call one person, there's a fifty-fifty chance they'll pick the winner (if they actually pick).  If you called everyone, you'd be pretty certain you knew who was going to win.  Somewhere between one and everyone is a manageable number that will give you a decent idea. 

Rasmussen picks 500.  Strategic Vision picks 800.  All other things being equal, Strategic Vision has the more accurate poll. 

But the overarching lesson of this bevy of polling should be what I already breezed by: It tells us nothing.

A lot of these "likely voters" will never make it to the polls.  A significant portion of the population hasn't even started paying attention (by the way - has ANYONE seen a TV ad for either candidate?).  A Eagleton Institute poll in March found the following:

more than 20 percent of voters said they had not heard of either candidate.

Twelve percent of voters said that Kean is or has been governor of New Jersey, while 3 % said the same of Menendez. Eleven percent said Kean was a member of the 9/11 Commission, while 6 % said the same of Menendez.

Of course, neither statement is true for either man.  All that is happening is that partisan Democrats are falling in line behind Menendez and partisan Republicans are falling in line behind Kean.  No one else, at least at this point, cares.