Let me say up front: I am not picking on USA Today, and this is not a blog about politics - it is about polls. So let's look at the numbers.
According to the article, the poll asked 1033 adults, and they claim that this sample is statistically representative of the entire country's opinion within 4% points.
The current population is 307,000,000, according to public data. [1] So this poll represents the opinion of 0.000336482% of the population. Personally, that doesn't sound very representative. Too many zero's in that number? Let me express it another way.
To determine the opinion of the country, they asked 1 out of every 297,000 people. So using these statistics, I could go to New York City and ask 27 random people their opinion, and have a statistically accurate representation of what everybody in NYC believes. Bringing that closer to home, I would only have to ask a single person in all of Northwest Arkansas to know how our region feels.
Opinions vary by what part of the country you live in, so let's assume they called people from every state. With only 1000 people total, they could have only called 20 people from each state. How do you feel about the idea that 20 random people from your state accurately represent the sentiment of your entire state?
To get a valid opinion of the country as a whole, I would think they would have to ask people from various demographic groups. To keep this simple, I am going to assume that this very short list represents a complete sampling:
- Gender: Male or Female
- Marital Status: Married, Single, Divorced
- Employment Status: Student, Hourly, Salary, Self Employed, Unemployed, Retired
- Income: Assume 6 levels of income ranges
One last thought. The article also pointed out that the survey was done to both land lines and cell phones. This is in response to recent complaints that younger and/or more technically savvy people do not have land lines, and therefore surveys that only use land lines are inherently skewed. So USA Today is trying to show that this was not an issue in their survey. But who these days is going to answer an incoming call on their cell phone from an unknown or blocked number? Does this really prove that they are reaching just as many college students, young active professionals, single moms, and third shift workers as they are reaching other groups like retired senior citizens?
This is why I do not pay any attention to polls like this, except to always read the footnotes. In the mean time, there is definitely a lesson to learn here: if you ever get asked to take a poll, DO IT, because your opinion will be counted as 300,000 people!